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0. Introduction 


This report draws together the diverse strands involved in developing a unique computer-based system to 
stage and manage Hodgkins Disease (HD). Those of us who sete on the final version of this project 
included two hematologists, a computer scientist, and a statistician. We have all contributed to this report, 

and our respective sections reflect not only our own special interest in and approach to the problem, but also 
the type of readers with whom we hope to communicate. We ‘expect that many will not read this report “in 
toto”, but that others working on similar problems may find particular sections helpful and interesting. In 
addition, because the program is not being maintained, we have included much data to insure that 


information from our extensive patient database is not lost. 


The report begins with a general discussion of the principles of management of HD, covered in sufficient 
detail to explain the problem to those not familiar with this field: “This section is aimed towards the 
non-physician or the physician who is not involved with HD managemeiit: it explains the rationale for a 


structured approach to the problem. 


Section 2 describes the patient database, and how this is used to predict the likelihood of the various stages of 
HD. A general discussion of Bayes Rule then leads to a description of the particular way this rule is applied 


to revise the probabilities of pathologic stage prior to management decisions. 


Section 3 deals with the technique of decision analysis. No previous Knowledge of decision analysis is 
assumed, and detailed descriptions of all the steps invotved are given, with the examples based on the HD 


system. The latter part of this section requires an understanding of Bayes Rule as described in section 2. 


Sections 4 and 5 are concerned with the statistical validity of the program. After describing the assumptions 
made in developing the program, this section deals with the way both these assumptions and the general 


validity of the program were tested. Details of the methods used are supplied, and wider application of the 
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method is discussed. . 


Section 6 is included to show exactly how the Bayesian and decision analysis techniques are applied. It gives a 
detailed description of a typical patient session, including all the special features which can be used to tailor 


the decision plans to a particular patient and the experience of a particular hospital. ; 


In contrast, sections 7 and 8 describe more general applications of our system. We use the program to 
approach some important management dilemmas in HD and we draw conclusions pertinent to large number 
of patients. Although these sections contain information mainly of interest to the practising hematologist, the 


discussion of decision-making thresholds has a more general application. 


Rigorous application of Bayes Rule and decision analysis to the inexact science of clinical management is 
bound to result in some interesting problems. These are addressed in section 9, in which we discuss the 


limitations we found in this approach. 


Section 10 is written as a guide to computer programmers interested in the implementation of such a program. 
The program is described in sufficient detail to enable a similar program to, be set up. Certain original and 


sophisticated techniques developed for this program are described. 


Finally, there are four appendices. The first is a comprehensive list of prior probabilities for the basic clinical 
findings, the conditional probabilties which are used to modify these prior probabilities, and posterior 
probabilities for every combination of clinical findings, all of which are derived from our 1200 patient 
database. We feel that these will be very valuable to the physician managing HD. The second appendix gives . 
the general format of the statistical test used to evaluate probabilities of stage as described in section 5. The 
short Appendix 3 documents the information used in the program to derive the conditional probabilities of 
stage from the lymphangiogram, and shows how these probabilities are derived from the data. Finally 


Appendix 4 gives an annotated trace of the decision tree analysis program in operation. 
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1. Hodgkins Disease - Modern Approaches to Staging and Treatment 


Although Hodgkins Disease (HD) is a-malignant disease of the lymphoid system, a malignant lymphoma, its 
| biologic course is different from the other lymphomas, and from neoplasms in general. In the majority of 
patients the disease arises in a single lymph node area, typically cervical, and spreads by lymphatic channels to 
involve contiguous lymph node areas in an orderly way. Local spread of disease to structures contiguous to 
affected lymph nodes is noted, particwuanly with involvement of mediastinal and hilar lymph nodes; 

sceasionally the disease may begin in a localized extranodal site. Splenic davolvcahet usually occurs as the 
disease progresses, and is thought to result from hematogenous spread, since the spleen has no afferent 
lymphatics, Hematogenous spread to other extranodal structures such as bone marrow, liver or lung does not 


usually occur until late in the disease. 


The extent of the disease determines the type of treatment which is indicated. To achieve uniformity in 
describing this extent the Ann Arbor seine system was adopted in 1970, and is now well-established 
(Carbone et al, 197 1) (Table 1.1). This stage of the disease, together with the presence (B) or sce (A) of 
the specific symptoms is the most important guide to prognosis, with the outlook becoming less favorable 


with advancing stage, and if symptoms are present. 


There is now good evidence that proper treatment of HD in certain presentations results in long term 
remissions and probably cure. For localized disease radiotherapy may be curative in a high proportion of 
patients if all the tumor is encompassed with a tumoricidal dose of radiation - usually in the vicinity of 
3500-4000 rad; for disease which has spread beyond the lymph nodes and spleen, combination chemotherapy. 
has given a high proportion of ‘patients such prolonged remissions that they may.be considered cured. For 
other stages, where neither radiotherapy nor chemotherapy when given alone has proven extremely 


successful, combinations of these two modalities are being tried to effect a cure. 


The diagnosis of HD often marks the beginning of a scrics of increasingly invasive diagnostic tests to 
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determine this stage in order to select treatment. This intensive staging may include an intravenous 
pyelogram, percutaneous bone marrow and liver biopsies, a gallium scan, a lymphangiogram, and 
computerized tomography; since all these tests have an appreciable false negative rate, and, with the 
exception of the biopsies, all have a false positive rate, (Table 1.2) the series often culminates in an exploratory 
laparotomy, with extensive lymph node sampling and splenectomy. If the extent of disease is established by 
laparotomy, or by histological confirmation of disease in extranodal structures by a biopsy, the patient is said 
to be "pathologically-staged”. Staging without exploratory laparotomy, where biopsies of extranodal ss are 


negative for disease, is termed “clinical staging”. 


Our basic purpose in applying the techniques of decision analysis to HID has been to tailor staging 
investigations to the individual patient, and thus to use the minimum number of investigations to select 
treatment. This has involved studying a large series of patients in order to draw general conclusions about the 
management of HD which could then be accessible to a wide audience of physicians, with or without access to 


computer facilities for individual patient study. 
For any patient the value in performing an investigation is related to three fiictors: 


1, the likelihood that the patient has the abnormality for which (s)he is being tested. 
2. the sensitivity or true-positive rate of the test. 


3. the specificity or true-negative rate of the test. 


All too often in HD the whole battery of available tests are used without analyzing their value in a particular 


patient, and without considering that they may provide overlapping information. There are a number of 


compelling reasons for performing the minimum number of tests required: 
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Table 1.1. The Ann Arbor Staging System for Hotes 's Disease 


STAGE CHARACTERISTICS 
Stage I Involvement of a‘single lymph node region, or of a 
single extralymphatic organ or site (designated by E). 


Stage II Involvement limited to one side of the diaphragm either 
of two or more lymph node regions, or localized-involvement 
of an extralymphatic (E) site and one or more lymph node regions. 


Stage III Involvement of lymph node regions on both sides of the 
diaphragm, which may include tocalixed invetverntent of an 
extralymphatic organ or spleen. 


Stage IV _Diffuse or disseminated involvement of one or more 
extralymphatic organ, or any liver involvement, with or 
without associated lymph node invol vement. - 


Each case is further classified as: 
A if asymptomatic 
B if any of the following are present: 


- unexplained weight loss of more than 10% 
of body weight in the preceding 6 months. 
- unexplained fever; a nee aC 
~ night sweats. 


modified from Carbone et al 1971. 


Table 1.2. False Positive and False Negative Rates of Tests . 


FP(%) FN(%) REF 


Gallium scan (abd.nodes) 5 49 McCaffrey(1976) 
Lymphangiogram 13 16. Appendix III 
Liver scan (abnormal uptake pattern) 23 38 Mildner(1973) 
: Ee _,,... Lipton(1972) | 
Spleen scan (i) filling defects SL Sitverman(1972) 
one Mildner(1973) 
(ii) size >15cm 7 .0 © 
Computerized tomography (abd. nodes) W..23B |., Lee(1977), Best(1977) 
_ ae Alcorn(1977) 
Breiman(1978) 
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1. with the exception of the gallium scan and computed tomography, afl the tests have a mortality 
risk. This may be extremely remote in te case of a bose marvow biopey, but there is a sul 
mnortality risk for such tests as the fymphangiogram aad laparoscopy, and an appreciable risk for the 
staging laparotomy. Data collated from the medical literasare for 2345 patients who underwent 
staging laparotomy is summarized in Table 1.3, and shows a mortality rate of 0.5% overall for the 


laparotomy and its immediate post-operative peciod. 


2. all tests, again with the above exceptions, have a definite risk of morbid complications; here too 
the highest incidence of serious conaplications is saa: fiat oiipiag:lnceric where 6.7% of 
patients experienced major complications as serious as pulmonary -embolism, subphrenic Pome 
intestinal obstruction. etc., and a further 15% had tess severe compligaticas.. This data is also 


summarized in Tabie 1.3. 


3, All tests are expensive. A full work-up which uses all the tests could cost in the vicinity of 


$10,000. Individual costs for the different procedures are given in Table 3.3. 


Information on the specificity and sensitivity of the tests used ta stage HD is readily available in the medical 
literature. Information on the likelihood of positive findings for the various tests under different conditions, 
however, is not. | 

In the following section we describe how we derived this information from data obtained from 
laparotomy-staged patients. We also describe a from the same data, we use Bayes’ Rule to predict the 
probabilities of the different stages for a particular patient with Hodgkins disease, s0 that decisions about 
"staging investigations and treatment may be made wsing the technique of decision analysis. | 
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Table 1.3. Mortality and Morbidity from Staging Laparotomy for Hodgkins Disease 


Reference No. of No. of No. of 
- Patients Deaths Major 
Complications 
Aisenberg et al (1974) 100 0 2 (2%) 
Andrassy et al (1977) 16 0 1(1%) 
Beretta et al (1976) 110 0 1(1%) 
Brogadir et al (1978) 90 0 16 (18%) 
Bruntsch et al (1977) 215 07 28 (10%) 
~ Cannon et al (1975) 400 1 15 (3.7%) 

Coleman et al (1976) 31 1 7 (23%) 
Ferguson et al (1973) 31 1 1(1%) 
Gamble et al (1975) 139 1 5 (4%) 
Garcia et al (1971) 20 0 0 (0%) 

Gazet (1973) 65 1 4 (6%) 
Hermreck et al (1975) 50 0 13 (26%) 
Jelliffe et al (1970) , , 2 0 1 (5%) 
Lowenbraun et al (1970) 122 0 0 (0%) 
Marston (1972) i) 2 4 (7%) 
Meeker et al (1972) 30 2 4 (13%) 
Mitchell et al (1972) 45 1 3(7%) 

Paglia et al (1973) 51 0 6 (12%) 

Piro et al (1973) 114 1 6 (5%) 
Poulsen et al (1977) 91 0 5 (5%) 
Prosnitz et al (1972) 40 . 0 4 (10%) 
Roberts et al (1976) 82 0 3 (4%) 
Rozman et al (1973) % 1 2 (4%) 

Smith et al (1972) 70 0 2(3%) 
Sutcliffe et al (1976) . 8 0 19 (1%) 
-Urlaub et al (1979) 107 0 5 (4%) 
Zarembok et al (1972) 30 0 0(0%) 


Total 2345 11 (0.5%) 157 (6.7%) 
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2. Deriving Probabilities of Stage 
2.4 The Database 


A great deal of information about the patterns of spread of Hodgkins disease (HID) has been amassed in the 
decade since exploratory laparotomy has been regularly used to stage the disease; from this information has 
come our knowledge of the predictable pattern of spread of HD according to normal lymphatic distribution. 
The different histological subtypes have been found to have diffrent pattems of spread, with the degree of 
dissemination increasing from lymphocyte predominant (LP) to nodular sclerosis (NS) to mixed cellularity 
(MC) to lymphocyte depleted (LD). We know also that the presence of symptoms is related to the spread of 
disease, with symptomatic (B) patients having more advanced disease than asymptomatic (A), Other factors 
known to affect the extent of the disease are the sex and the age of the patient. The younger age group, 
corresponding to the first incidence peak in the characteristic biitjodal age:incidence curve'of HD, shows an 
almost equal male:female. ratio, a predominance of the NS histologic pattem, a higher, frequency of 
mediastinal involvement and a more benign clinical course. In contrast, the older group, corresponding to the 
second peak of the age:incidence curve, shows a higher male:female ratio, and a number of inter-related 
clinical features including greater proportion of the MC hitiglogical type, a much higher incidence of 
infradiaphragmatic involvement at presentation and a more aggressive clinical course (Gutensohn et al 1977). 
The pattern of involved supradiaphragmatic lymph nodes is related to the frequency of associated 
infradiaphragmatic disease; left cervical node involvement is associated with a higher incidence of disease 
below the diaphragm whereas with mediastinal involvement a lower incidence is seen. 

We have used data from approrimately 1200 paticnts who have been pathologically staged, to help predict the 
“likelihood of the various stages of Hodgtins disease according to the ‘basic clinical data known about the 


patient. 


All information about the following findings which are available for a given patient have been collected and 


stored: 
Histologic subtype 
Presence (B) or absence (A) of symptoms 
Sex 
Age 
Specific supradiaphragmatic lymph node groups involved 
Alkaline phosphatase levels 
Clinical splenomegaly 
Absolute and percentage lymphocyte count 
Liver/spleen scan 
Percutaneous bone marrow biopsy 
Percutaneous liver biopsy 
Gallium scan 
Computer-assisted tomography (CAT scan) 
Lymphang jogram . 
Per itoneoscopy 


Laparotomy 
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A typical string of patient data may be seen below: 


((SOURCE TUFTS) (ID XX) (AGE 20) (SEX FEMALE) (HISTOLOGY MS) (A-OR-B A) 
(LOCATION-PRESENTING-NODES (LEFT-NECK RIGHT-NECK)) 
(MEDIASTINUM NEGATIVE) (LIVER-SCAN NEGATIVE). . 
(SPLEEN-SCAN NEGATIVE) (SPLENOMEGALY NORMAL) 
(ALK-PHOS (BODANSKY 3.0)) (GAL POSITIVE-ABD-NODES) (LAG POSITIVE) 
(BMBX NEGATIVE) (LBX UNKNOWN) 
(LAP ((LAP-SPLEEN NEGATIVE) 
(LAP-MARROW NEGATIVE) 
(LAP-LIVER MEGATIVE) 
(LAP-NODES POSITIVE)))) 


Patient data from the following institutions was obtained: 


Tufts-New England Medical’ Center 91 patients - 


Joint Center for Radiation Therapy 89 patients 
Massachusetts General Hospital 113 patients 
Stanford Division of Radiation Therapy 504 patients 
Harvard School of Public Health a “$b patients eo" 7 


The remainder came from individual patient data in the medical literature by Bearman et al (1978), Hanks et. 
al (1971), Jeiliffe et al (1970), Lowenbraun et al (1970), Mitchell et al (1972), Prosnitz et al(4912),:Zarembok 


et al (1972). 


The padtont with is tunccarmnon ln treciepmragesaiic prcarseyeon of Hodgkins Cocae was poorly tcpirscnen 
in the initial database. Furthermore, in this type of patient certain test results have a different meaning in 
terms of staging, ¢.g. such findings as involved abdominal lymph nodes or involved spleen define the patient _ 
as having stage II discase only. For these two reasons, we have excluded the patients with an 


infradiaphragmatic presentation from the database, and do not attempt to analyze this type of case. 


Initially the proportion of patients in each pathological stage was calculated from the raw data. Stages I and II 


were analyzed together, as therapeutic decisions for these stages are almost always identical. Each finding was 


then assessed to see whether it was helpful in predicting the pathological stage. Certain findings - alkaline 


phosphatase and clinical assessment of splenomegaly - were not found to be helpful in this respect, so they 


were not used in the final version of the program. 


For those findings which were found to be helpful in distinguishing among the different stages, the number of 


_ patients in each pathological stage was ascertained and was expressed as a proportion of 1.0 e.g. for the 


finding Histological Subtype: 

Histological Pathological 

subtype stage 

LD ItIl 
Ill 
IV 

Lp , I+II 
II! 
IV 

MC III 
IItl 
IV 

NS ItII 
{II 
IV 


No of 
patients 


Proportion 
of patients 


0.32 
0.41 
0.27 


0.75 


0.22 


0.03 


0.42 
0.44 
0.14 


0.62 
0.32 
0.06 


One can use this data to calculate the probability of being in a particular pathological stage, given the 


histological subtype, e.g. the probability that a patient demonstrating the NS subtype will be in stage III is 


0.32, or simply stated P(IIIINS) = 0.32 


Similarly the conditional probabilities of STAGE given SYMPTOMS can be obtained from the database: 
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Pathological Me of . Prebability. 


stage patieats 
ry Ietl 485 9.86 
U1 226 9.3 
Iv 32 0.84 
8 Ie] 181 o.41 
{il 187 0.42 
Iv 63 0.17 


From this one may derive the probability of STAGE given SYMPTOMS eg P(IVJA) = 0.04. Data can be 


similarly expressed for afl the findings entered. 


If the findings were all independent of one another, then one could find. the probability of having a 
combination of findings in stage (1+ ID), stage III, or tage TV for any patient'sinply by multiplying together 
the conditional probabilities of each finding given the stage. For example, for’an asymptomatic (A) male 


patient who has the MC histological subtype: 


p(A MALE MC|III)’ would be p(A| IIE) X p(MALEJIII) x pve ta) 


Clinical information, however, suggested that some findings were likely to be idterdependent. Chi-squared 
testing (see Chapter 4) did confirin this, and dependence was shown among the the patient's sex, the presence 
or absence of symptoms and the histological subtype, though. it was aot seen among the other findings used 
by the program. This dependence precluded the simple type of calculation shown above. Instead, prior 
probabilities of stage were calculated from the data forthe triad of interdependent findings ~ the patient's 
sex, the grescace or absence of symptoms aad the histological subtype ~ making sixteen different 
combinations of prior findings. For aay patient, Bayes’ rude was then weed to modify these prior probabilities 
by the conditional probabilities of FINDINGISTAGE for the subsequent findings used in the program -- the 
"patient's age, involvement/not-involvement of cervical and mediastinal lymph nodes, and the results of the 


various tests, including liver-spleen scan, bone marrow and liver biopsies, gallium scan, and lymphangiogram. 


2.2 The Use of Bayes Rule To Revise Probabilities of Stage in Hodgkins Disease ae 


Bayes Theorem is a mathematical method for modifying probabilities when new information is available, We 
have used this method to revise the probabilities of stage, as we take into account the basic clinical 


information known about the patient and the results of testing. See section 2.2.2. _ 


2.2.1 General Application of Bayes Rule 


Consider a test with two possible outcomes (T+ or T-) which is used to modify the probability of a patient 
being in a disease state, D. If P(D) is the probability of the disease state prior to the test, the revised 
probability P(D[T +) ie. the probability of D given a positive outcome, and of P(DIT-), i.e. the probability of 


D given a negative outcome, can be expressed as follows: 


P(D|T#) = P(D) P(T+1D 


P(D|T-) = 


Where P(T+|D) = Probability of a positive test, given the disease 
 P(~D) = probability of no disease — | 


P(T+|~D) = probability of a positive test, given no disease, i.e. 
the false positive rate. 


P(T-|~D) = probability of a negative test given no disease 


P(T-|D) = probability of a negative test given disease, i.e the 
false negative rate. 


Knowing the prior probability of a disease state, P(D), and the false positive and false negative rates of a test 


used to detect it, one can use Bayes’ rule to revise P(D) according to the outcome of the test. - 
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222 Use of Bayes Rule im the Hodgkins Disease Program 


in our program we have used Bayes Rule to reviee the probabilisies of cach of the stages of HD, taking into. 
account both basic clinical information known about a patient aad the results of any tests. Prior probabilities 
for each stage are those derived directly from the database for the three basic findings which have been found 
to be interdependent, namely histologic subtype, sex, and the presence or absence of symptoms. Prior 
probabilities for each of the sixteen possible combinations of the findings are givea in Appendix I. 


To demonstrate how we apply Bayes Rule we will show the calculations involved in the case demonstrated in 
Chapter 6. This patient is a male with B symptoms - weight loss - and the nodular sclerosis hisinlogi 
subtype. The prior probabilities of stage for this patient are: 


Stages I+II 0.34 
Stage III 0.66 
Stage IV 6.16 


We wis o revise these probabilities of tage knowing the pleat isi the age group 1-39 yar For this 
we need to know the conditional probabilities of AGHISTAGE which have been cakulated from the 


Spb mesy 


database, and which have been shown wo be independent of other findings Tete conditional probebtes 


are: 
Younger than 12 years +11 0.93 HE 205. IV 9.08 
12 to 39 years I+il 6.84 III 6 IV 0.46 


Older than 39 years  A#EL 0.14 .1EJ. 0.23 | IV 0.47 


To update the prior probability of stage (1+-ID) for example 


p(I+I1}AGE12-30) = 
p(1+11) .p(AGE12-39] ( 1+11) 


OO Am ee Oe Oe Oe mae ee Oe ene an eS OS De DOD OE Se OE OP OD Oe Oe MO Oe UD AD OE Bie OD Om EOD Oy Om hem ab ew Hate OD ED OD OD SD ED ED OD we OS > OD OD ate ED aD am oD ee OD aD wD Dt 


p(I+I1).p(AGE12-39|(1+I1) + p(III).p(AGE12-39{(I11) + (IV) .p(AGE12-39] (IV) 


The term p(I+1l).p(agc\(1+11)) corresponds to the “truc. positive” if one considers age in the sense of a 


"test", with being in the 12-39 age group a “positive” result and the other age groups a “negative” result: This . 
makes the other terms in the eqation - p(III).p(AGE12-39( Hi) and pUV).(AGE12-39{TV)- the equivalent of 


false negative results. Substituting the probability results in the equation: 


p(I+I1)|AGE12-39 = 0,34 X 0.84 


een eet ee reer ttadoeace ecw ue ces 


(0.34 X 0.84) + (0.6 X 0,74) + ((0. 16 X 0.45) 
= 0.3926 


Similar calculations are carried out to revise the prior probabilities of stage III and stage IV according to the 


conditional probabilities of agelstage. 


These posterior probabilities now become the new "prior" probabilities which can then be revised by the next 


Piece of information available, such as the specific sites of supra-diaphrag patic lymph node involvement, 


Bayes Rule is applied to the tests used in staging HD according to the general formula given in section 2.2.1. 


For example, suppose we have calculated that the probability of abdominal node involvement in a patient 
with supradiaphragmatic HD is 0.30. A gallium scan is found to be positive for abdominal nodes, and we 


wish to know how this positive result has modified our initial probability. 
P(+NODES) = 0.30 therefore p(-NODES) = 0.70. 


From the literature we find that when the gallium scan is-used to detect involved abdominal nodes in HD: 


False positive rate = p(+test|-NODES) = 0.10 
False negative rate = p(-test|+NODES) = 0.50 


where the symbols + and - are used for positive and negative, i.e. 
+NODES are lymph nodes involved with HD. 


From these we deduce: 
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Frue positive rate = p(+test|NOBES) = &.68 

Teve negative rate = p{-teet|-NOBES} = €.9¢ 

We can now caltutnte the revised probability of abdominal node invobrement:. 
p(+HODES) pf{+test | NGBES) 


p(+NODES|+test) = ----------------------------------------------------- 
p(+ODES) -D{+best [+EQGES) + p{-HOBES) .p(+test[-NODES) 


(.3 X .6) + (.7 X .10) 
= 0.68 or 68% 


‘The use of Bayes Rule here has revised the probability of abdominal node involvernent upwards; from 6:30 to 


0.68 in the light of the positive result for gallium scan. 


In our application of Bayesian methods to the staging of Hodgkins disee, we use the tests to modily 
information about a stage of the disease, rather than the specific sites of involvement, as demonstrated in the 
example above. If we wish to find the revised probability of stage IIT disease in a patient, knowing that the 
gallium scan is positive, we can write the basic calculation a: 
p(*GAL] IIE). p( III) 


| PEELIP+GAR) © ~~-~newnwneene sewer cee es ae ell eee 
: p(+GAL}I+II).p(I+IE) + p(+GALEIII).p(IIZ) + p(+GAL| IV) .p(1IV) 


‘However, the literature does not contain data in the form of conditional probabilities of test results in certain 


stages of the disease. This calculation, therefore must be carried out in two-stage: 


P(+GAL| IIT) = p(+GAL]+NOBES} p( +8OGES] FIT) 
TRUE POSITIVE 


, : 
p(+GAL|-NOBES) p{-NOQDES| III) 
FALSE POSETIVE 


‘The terms for p(+GALII+II (or TV) are expressed in the same way. The probability of positive and 
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negative nodes in stage III is obtained from the database, i.e. 


In stage IIIA: 


p (+ABDOMINAL-NODE- INVOLVEMENT ) 
p(-ABDOMINAL-NODE- INVOLVEMENT ) 


“ow 
o 
oJ 
a 
nN 


These lead to the conditional probabilities: 


p(+GAL|III) = 0.6 X 0.62 + 0.10 xX 0.38 
TRUE POSITIVES FALSE POSITIVES 


0.31 + 0.038 
= 0.36 


That is the probability of finding a positive Galtium scan in a stage IIIA patient is 0.35. 


Similar calculations are performed for all stage/symptom combinations for this test and all the other tests 
used, so that conditional probabilities of the test given stage and symptoms are available for the Bayesian 


calculations. For the gallium scan these are: 


NEGATIVE GALLIUM SCAN 


I+II A 0.90 8B 0.90 
III A 0.65 B 0.66 
IV A 0.61 B 0.54 
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23 Defining the Age Groups 


‘Comparing the incidence of Hodgkins disease with the age at presentation shows a bimodal age/incidence 
curve with one peak in early adult life, and a second sustained incrésse in incidence from the fifth decade. 
The two age groups defined by this curve show important differences, with the age at presentation showing 
correlation with the extent of the disease. The younger group shows an approximately equal male:female 
ratio, a predoeninsace of the nodular sclerotis (NG) bistohogic patiern apd a moet benign clinical couree, with 
mediastinal involvement quite common; in contrast, the oer group shows a higher male: female ratio, and a 
number of inter-related clinical features including a greater proportion of the MC histological type, a much 


higher incidence of infradiaphragmatic involvement at presentation and a more aggressive clinical course. 


We have, therefore, used age as one variable for esimating the probabilities of sage in Hodgkin's disease 
patients. Conditional probabilities of AGE given STAGE are used in applying Bayes rule to calculate 
posterior probabilities of STAGE given AGE. ‘We of is ori gues he umpton that ap is 
independent of the findings used to calculate the prior probabilities” This assumpt 
arbitrarily selected age groups - 0-15 years, 16-30 years, 31-45 years, and older than 45 years - initially used in 


was checked for the 


the program by means of Chi square analysis, and found tenable (Eisen 1977). 


Subsequent analysis of the larger database showed that there were essentially three age groups distinguished 
by distribution of stage: 0 to 11 years, 12 to 39 years, and 40 years and over. Homogeneity was observed 
within each group, with some minor statistical fluctuation, and distinct differences were observed between 


groups (as determined by Chi square testing). 


3. Decision Analysis 


Decision Analysis is a technique for evalneans complex decisions by ekg them down into their simplest 


component parts, quantitatively evaluating teas then reassembling them in a: a logical ai 
Decision analysis involves four sequential steps: 


a) constructing the decision tree which displays, int’ chronological order, all the options for a 
particular decision, and every possible outcome of eath option, whether determined by chance, or 
by further decision-making. 

b) evaluating the likelihood of those gutcomes in the decision tree which are eeermined by chance. 
c) scoring each possible open. acotding to a system of utilities. 


d) evaluating all decision branches and selecting those with the highest utility. 
- 3.1 Constructing the Decision Tree 


The decision tree is displayed on a branching framework, and includes all the options among which one must 
choose, with all possible outcomes of each option displayed. . Subsequent outcomes include both those 


determined by chance, and those which. involve later decisions. 


At each branch point in the tree where a decision must be ‘matie, a ‘shuaff‘square is used to denote a decision 
node. At any point in the tree where the outcome of # decision is determined by chance, a smafl circle denotes 


a chance node. 


Below is shown the simplest type of decision tree used in this work: the basic TEST versus TREAT decision, 


in which a physician has to decide whether to perform a test to define the patient's disease hore accurately or 


ova 
2A 


whether to treat (Rx) the patient without further ado. By convention, the decision tree is drawn with the early 


decisions on the left, the later outcomes on the right. 
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At decision node a, the decision is between treatment (Rx) or performing TEST-1. At chance node b, the 
patient may die (+) from the test - this risk being the observed test mortality rate - or may survive the test. At 
chance node c, having survived the test, the patient may have ether a positive or a negative result for the test 
At each decision node d, one must now, in the light of information, gained from the.test, make another TEST. 


versus TREAT decision, for some other test. 


The actual decision tree involved in the selection of test procedures and treatments in. Hodgkins disease is 
very complex, because: 
(i) there are a number of different tests among which one must choose - namely bone marrow 


biopsy (BMBX), percutaneous liver biopsy (LBX), peritoneascopy with guided liver biopsy (PTX), 


gallium scanning for abdominal node involvement (GALL), lymphangiogram (LAG), and staging 
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laparotomy (LAP) - and one may choose any number of these. 


(ii) The order in which the tests are performed may be important. A test sequerice stops witha ~ 
positive bone marrow or ver biopsy which constitute definite evidence of disseminated (stage IV) 
disease, or with the staging LAP, which is considered for practical purposes as “perfect: 
information", i.e. it is assigned neither a false pen nor a false negative rate and its results are | 
always final. Following a negative bone marrow’ or "Tver biopsy, or after gallium scan or: 


lymphangiogram further tests must be écntldeced 


Below we show part of the decision tree relating to the selection of tests, to demonstrate the recursive way.in. 
which any tests not yet performed are considered. This decision on displays only one of the ‘branches which 
occur at each decision node, following the initial decision to select bone marrow biopsy. Where further 
decision “branching” follows, the test name is followed by dots, eg. "GALL..." . A similar branching 


structure exists for each of the other initial tests, except for laparotomy, where results are final. 


The actual tree evaluated is even more complex since at every decision node, the decision involves not only 
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farther TESTING, but must also include an evaluation of the TREATMENT option. The actual framework 


of the tree is shown below: 
LAP 
LAP 
LAP -_ LAG 
- ¢} at 
[F-O Rae ean 
: 3tF p LAG 
Ff: ax + a 
5 GALL L Rx : 
+ 
GALL 
LAG 
LAG 
Rx 
Rx 


The TREATMENT option may include any of a number of different treatioents; these include radiotherapy 
to all lymph-node areas, known as total nodal irradiation CTND, radiotherapy which spares the pelvic lymph 
ses ksh Kone Gi easaded meats Wiadiadine GAO es the comibinstion chemothesipy regime 


The TREATMENT branches for asymptomatic (A) and symptomatic. (B) discase are shown below. For A — 
disease there are three treatments to choose amongst, for B patients only. two in our basic decision tree. The 


tree may easily be modified, however, to include other treatments. 
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patient 


patient 


It is also possible to compare single modality therapy, with regimens which include both irradiation and 
combination chemotherapy (combined modality therapy). If this option is selected, however, for. proper 
comparison with the single modality therapy, allowance must be made for “salvage” therapy (usually with 
MOPP) should relapse occur. Below we show the treatment branch when combined modality therapy is 


included in the treatment being evaluated for patients with symptomatic (B) disease. 
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Extended: A Pena 
No 
™ pK mieaic 
Relapse => MOPP e es ; 
ie ssdinisiadl rermiasion 
spp | rem aalon 
{ ) cs ae 
Relapse => MOPP 
& 
No 
Combined 


3.2 Evaluating the Likelihood of the Outcomes 


In this program we start with those probabilities of the pathologic stage derived from the repeated use of 
Bayes Rule, according to the basic findings known about the patiest. For example, for a 20 year old 
‘nodes and the mediastinum, and who has a normal live/spleen scan, the prior probabilities are modified as | 
shown in Table 3.1 before decision analysis is used to decide Further management: 


For this patient, these probabilities of stage derived from the basic clinical findings now become the “prior 


__ probabilities” for the decision analysis. As cach test is evaluated, these probabilities are modified by: 
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Table 3.1. Results of Bayes Rule, Incorporating Basic Clinical Findings 


Finding Stage I+II Stage III Stage IV 
M WC B 0.20 0.44 0.36 
Age >11<40 years 0.26 = 99.60 - 3s O24 
L cervical nodes -ve 0.30 0.46 0.24 
R cervical nodes +ve 0.27 0.42 0.31 
Mediastinal nodes +ve 0.30 0.40 ° -; 0.30 
Spleen scan normal 0:43 - 0.34 0.23 
Liver scan normal 0.48 9.37. «© °° 0.16 


1 the known mortality rate for the test (see decision node b in the first basic TEST versus TREAT 


decision tree in section 3.1). 


2. the known false negative and, where applicable, false positive rates for the tests, applied 


according to Bayes Rule. 


Suppose in the example, we are considering the branch of the tree which begins with bone marrow biopsy; 
note that this test has. no false positive rate, and from the reported false negative rate, we have derived the 


following conditional probabilities: 


Probability of NEGATIVE bone mre biopsy (-BMBX) with 
I+1IB = 1.0, IIIB = 1.0, IVB = 0.66 
Probability of POSITIVE bone marrow biopsy (+BBX) with 
I+1IB = 0.0, IIIB = 0.0, IVB = 0.44. | 
The basic probabilities are modified by Bayes rule, using these conditional probabilities, e.g.: 
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= :  pciv) pt -BMBX | IV} 


Oe 8 ee Oe 8 Oe 2 ER OP OO ee ES OE OER COTTER Eee eee 


p(I+II) p(-BMBX}I+IZ) + p(1II) p(-BMBXPIII) + p(IV) a(-BNEXtIY) 


= 0.15 X 0.568 


0.48 + 0.87 + 0.084 0,034 
Snir calculation ae cared ot ox sage +1 an ving eed probabil ithe tet negate a 


I+11B Q. 51 TTB 0. » 40 ive (6. 08. 


a bop, 


IF pve, he et conse eine evidence of age IV deme, Thee rv probes ae clr 


iat he nei lage of the Géchiion Gee: 
ae CE 
—_ GALL . 

LBX eal —O 
; ge tase LAP 
§1:.40: aad © LAG | | Pe s 
=! BX : aes SEF Pu oa 
a @ cat L” ae Re 
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3.3 Assigning Utilities 


The utility is the value placed on a particular outcome. Utilities may be expressed on an arbitrary scale, or a 


measurable quantity may be used. We have used several different types of utilities to assess our decisions: 


i) Disease-Free Survival Five Years After Treatment (5 yr DFS). This figure is used because results of 
' treatment are very frequently reported in the medical literature in this form. In addition, in HD, very few 


patients relapse after this period, and the 5 yr DFS can be used as an index of cure. 


ii) Morbidity Units - in addition to its mortal risk, each diagnostic test and each treatment has an associated 
risk of morbidity. This includes both the risk of severe complications, such as pulmonary embolism after . 
staging laparotomy, and the duration of pain and incapacity associated with the test or treatment under 


normal circumstances. 


For each test and treatment we asked a number of physicians, directly involved in the investigation and 
treatment of patients with HD, for their estimate of the "pain and discomfort", and the "duration of 
incapacity”, for the “average” patient undergoing a given test or treatment. An arbitrary standard was made 
which defined the “pain and discomfort" of a bone marrow biopsy or of a single day’s incapacity each 
equivalent to 10 units. The literature was then searched for data on the complication rates of tests and 
treatments. Each complication was assessed according to the same criteria as the basic test; this figure, 


‘multiplied by the incidence of the complication is included in the morbidity values shown in Table 3.2. 


iii) Dollar Cost of Tests and Treatments. These are shown in Table 3.3. 


Each of these types of utilities is evaluated separately. At present, we have not found a satisfactory way to 
incorporate these disparate elements in the same scale of values, and no attempt has been made to "trade-off" 


onc type of utility with another. 
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Table 3.2. Morbidity Units for the Staging Procedures and Treatments 


Gallium scan 

Bone marrow biopsy 

Liver biopsy 

Lymphang iography 
Peritoneoscopy 

Laparotomy 

Extended mantle radiotherapy 
Total nodal irradiation 

MOPP chemotherapy 


7 units 

10 units 
26 units 
40 uaits 
80 units 
875 units 
1000 units 
1600 units 
3000 units 


Table 3.3. Dollar Cost of Tests Used in Staging Hodgkins Disease 


Test Professional 


Cost 

Liver-Spleen Scan 213 
CT Scan 

- with contrast 168 

- without contrast 118 
Gallium Scan 259 
Liver Biopsy 72 
Bone Marrow Biopsy 94 
Lymphang iogram | 421 
Laparotomy 


Fee 
60 
100 
70 
60 
155 
60 
276 


Total 


273 


268 
188 


319 
goo 
164 


606° 
" 8000-s000"* 


* includes 2 days in hospital, type and cross-match, ‘etc. 


** this includes 7-10 days in hespital, fees for surgeon, physician, 


and anesthesiologist, use of operating and recovery 


room, laboratory tests, X-rays, and ipathelegy fees. for. operative 


specimens. 
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3.4 Evaluating Decision Branches 


This step is known as “averaging out and folding back”, or in more technical terms “the process of backwards 
induction in the theory of dynamic programming". Starting with the most peripheral decision nodes, all 
outcomes of a particular “sub-decision" are evaluated, and the. option with .the highest utility ascertained; 

this value is then assigned to the decision node. For example, in the decision. tree shown below, one would 


initially evaluate each of three branches after lymphangiogram. 


To evaluate these, one must know the probabilities of pathologic. stage at that particular point in the tree, 
when the prior probabilities have been modified. by a negative BMBX result and by the result of 
lymphangiogram. We will use the symptomatic male patient from the example in Section 3.2. We will 


evaluate the branch indicated by the arrow: the probabilitics of pathologic stage for this paticnt at this 
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decision node, with negative bone marrow and positive lymphangiogram are: 


Stage I+ITI:. 0.22 


Stage III: 0.64 
Stage IV: 0.16 


When the patient’s stage is not known for certain, assessitig the utility of the option to proceed directly with 
treatment, involves calculating the average disease-free survival for each treatenent and choosing the 
treatment with the highest value. The utility fora particular treatinent is calculated as a weighted average, 
using the probabilities of stage and the five year disease free survival values for each stage for that treatment, 


e.g. assessing the value of total nodal irradiation for the patient: 


Stage Probability of 5-year DFS 


that stage for that stage 
with TNE - 
I+11B 0.22 times 70% = 16.4% 
IIB 0.64 times 26% = 16.0% 
IVB 0.16 times ox 0.0% 
Weighted average = 31.4% DFS 


This is then compared with the weighted average obtained from the corresponding values for MOPP 


chemotherapy: 
Stage Probability of 5-year DFS 
that stage for that stage 
with MOPP 
I+1TIB 0.22 times 60% z= 13.2% 
IIIB 0.64 times 50% = 32.0% 
IVB 0.15 times 35% = 5.25% 
Weighted average = 60.45% DFS 


In this example the weighted average for MOPP chemotherapy is greater than that for total nodal irradiation, 
- and would thus be preferred to TNI as the treatment option. The same techniques are used to evaluate 
combined modality therapy as one of the treatment options,’ but the calculations are more complicated 


because, as already described, one must alow for salvage of patients who initially reccive single modality 
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therapy. 


To assess the value of laparotomy, the probability of each stage, is multiplied by the best Treatment for that 
stage, e.g. to assess the utility of laparotomy before any other tests have been performed, again a weigtited 
average is used as follows: 

' Stage Probability of 5-year DFS 


that stage for that stage 
with BEST Rx 


I+IIB 0.22 times 70% THI = 15.4% 

IIIB 0.64 times 50% MOPP = 32.0% 

IVB 0.15 times 35% MOPP = 6.25% 
Weighted average = =_- 62.65% 


To obtain the true utility for laparotomy, one must also take into account its mortality risk - this is usually 
assessed as 1%, which includes 0.5% peri-operative mortality risk, and a farther 0.5% late mortality risk from 


post-splenectomy sepsis. The laparotomy branch of the decision tree actually looks like: 


Laparotomy mortality 


The value obtained in the calculations above for the average utility of LAP must therefore be multiplied by 
0.99 to. give the actual value which should be compared ‘with the best treatment option (52.65% X 0.99 = 


52.12%). When this modified value is compared with that derived for immediate treatment prior to any 
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testing, it is apparent that the laparotomy option (52.1%) still gives a higher chance on the average for 
discase-free-survival than the preferred treatment option MOPP (50.45%), so its value is assigned as the value . 


for that decision node. 


The other decision nodes which follow lymphangiogram are evaluated similarly, the only difference being the 


probabilities of stage which occur with equivocal and negative lymphangiograms. 


Positive §2.1% DFS (LAP) | 


Equivocal 61.7% DFS (LAP) 


Negative 63.8% DPS (LAP) 


To determine the value of the whole lymphangiogram branch - or the value of any branches after a chance 
node - one must first calculate the likelihood of each outcome after the chance node, and then multiply the 
expected utility of each branch by its likelihood. This gives the overall expected utility for the whole 


lymphangiogram branch. 


52:1% (LAP) = 219 


61.7% (LAP) = 11.1 


63.8% (LAP) = . 255 


0.40 


Expected DFS for lymphangiogram branch = 58.5% 
Calculations for the whole tree are carried out similarly: 
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Rx - apes 
(53.8, MOPP] 


p = 07 
0.0:0.0:1.0 


[52.6] 


When all the calculations are performed, the utility of the BMBX option is 56.9% DFS, which exceeds that of 


immediate therapy (52.6% DFS). . 
The best course of action in this situation is therefore: 


To perform bone marrow biopsy; if positive give MOPP, if negative further testing is 
necded. Regardless of the results of lymphangiogram, laparotomy has a higher utility 
than the treatment options. This indicates that lymphangiogram is not necessary for 


this patient, but that laparotomy is needed to select appropriate treatment. 


In summary, any decision trec is evaluated in the following way. Starting with the “leaficst” part of the tree, 


i.e. that furthest away from the fundamental decision, utilities are calculated in two ways: 
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1. at a decision node utilities are calculated for each branch and the highest one is selected as the 
value of the node. 


2. at a chance node utilities are calculated for each subsequent branch; the utility of a particular 
branch is then multiplied by its probability and all the products obtained are added together to give 
a weighted average which is the expected utility for that chasice node. 

In a decision tree as complicated as that for HD, where there are more than 2000 decision branches to 


evaluate, it is necessary to use a computer for the calculations. 
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4. Conditional Independence Assumptions 


For each patient with Hodgkins disease, we start out with information about histology, symptoms, sex and 
age. As further clinical information is considered, and diagnostic tests are. performed mare clues about the 
spread of disease are obtained, In the decision analysis program we repeatedly ask the question: given the 


information available so far, what are the probabilities of eich stage in this patient? 


Suppose we have a large number of past ciilenls whose stages are now known, and who are identical to the 
current patient in all eva respects. Probabilities about the new paticnt’s stage could be estimated directly 
from the actual distribution of stage among the ‘aitieg patents “Thus, for example, if 30% of the former 
patients were in stage III, 30% is a sensible estimate for the probabiy of stage I III disease for the patient at 


hand. 


As more and more findings are added to the database, however; the cohort of past patients highly similar to 
the present one becomes smaller. Probabilistic ibs arate obtained under the method just described would 
be subject to such large sampling error as to be almost aca If our calculations of staging probabilities are 
to be at all informative, certain independence assumptions must be fairly 2 accurate. These assumptions, 
crudely speaking, assert that the way a particular finding is used in-medifying a patient’s prognosis does not 


depend on any other information about the patient. © ~~ 


In this section, we discuss our initial assumptions of independence, the tests we performed to validate them, 
and a modification of the program that arose when one of the independence’ assumptions was found 


inconsistent with our data. 
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4.1 The Independence Hypothesis 


Suppose that, for a given patient, we have a set Y of prior findings and a new finding C. Bayes’ Theorem 


asserts that the probability the patient is in stage i (i = 1,2,3 or 4) given this inform 


P(Y|i) X P(C|4 and ¥) cee 
P(ifY and Cc) = &sSAe eros aa. oe ts we i as bn et i : xX P(7) “(4)- 
an 


where P(i) = probability of i, PTY) —‘conditional probability of i given Y: 


The mecependeore sumption we make is that re) and 11) = ACH, which means s that, among | those i in 
sage | tere i no coreltion betwen tose with fing Cand tose with fining “Under this 
independence assumption, Bastion (1) becomes: 

P(Y}i) X P(C]i) 


PC EPY and C) = cosceenseeeresg—ee XK PEER (1A) 
eC as¢ Cc) 


The vale sng equation (1A) nea of) thal awe se apr dat ses, et je sampling 


error, in the calculation of needed probabilities. 


Of course, if the independence assumption is false, calculations based. on it cannot be trusted. . Thus it. was 
necessary to perform statistical tests of the null hypothesis Hg that, PACH and Y). = P(CQi) where C and Y were 


varied. The tests we employed were tied to the familiar Chi-squared test. 
4.2 Tests of Independence 


For all patients with Hodgkins disease we start with information about histology, symptoms, sex and age. 
_ Our initial estimates of staging probabilities were based on histology and symptoms alone. Thus, for example, 
the data-based estimate of the probability of being in stage Il] given mixed cclfularity histology and the 


presence of symptoms (e.g. night sweats) was 0.30. 
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The data make clear that the very fact of being male makes one more likely to be in a later stage. Probabilities 
of stage given sex as well as symptoms and histology were estimated by the program under (1A), under the 
assumption that P(male given i) = ‘Pumale given i and Y), where Y is any one of the eight possible 
combinations of findings about histology and symptoms. The number 8 arises because there are four possible 


histologic subtypes and key symptoms are either present (B) or absent (A). 


The following example will show how we used the Chi-squared test to examine the null hypothesis of 
independence among the basic findings sex, nee and the histological subtype (Eisen, 1977). In our 
database, 51% of the patients in stage I are male. 164 of the patients are asymptomatic (A), of whom 132 have 
the Nodular sclerosis (NS), 17 have Mixed cellularity (MC), and 15 have the. Lymphocyte predominant (LP) 
histological subtypes. Under the independence hypothesis, the number of males in stage 1 who are also NS 
and A would be expected to be 132 X'0.51 = 67. ‘The expected number of males who are A and MC, and A 


and LP, would be 9 and 8 respectively. Consider the table below: 


Stage I Asymptomatic Patients 


Histology Expected number of Males (E;) Actual number 
Under Independence Hypothesds of Males (Aj) 

NS (132 patients) 67 60 

MC (17 . ) 9 11 

LP (16 . ) 8 13 


The viability of the null hypothesis depends on whether the observed differences between the expected (E) 
and the actual (Aj) numbers of males can reasonably be dismissed as chance fluctuations. We calculate the 
“discrepancy index” B, ‘ier the Tule Bj = (Ej - A) / & for cach of the Sana above. We make similar 
calculations for those males who did have symptoms, and also for the females in stage I. ‘The sum of the Bis 
for all categories is compared to some cutoff number Z, with the null hypothesis rejected if the sum exceeds Z 
and accepted if the sum is at or below Z. Z is dependent on the number of different categories, and the extent 
to which certain categories are "redundant" of others; both these considerations are very familiar to users of 


Chi-squared tests. 
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It turns out that, for sex vs. histology and symptoms, the independence assumption was not acceptable under 
the statistical test. Thus to include the finding “sex” in the prediction, we had to go back to Equation (1) _ 
csilies than aise thse akeaplee fora ia). We then preceeded in situtlar fashion’ to’ see ‘whether age could be 
treated as independent of sex, symptoms, and histology in‘ the estimation of staging probabilities. Here, the 


independence assumption passed its test easily. 


We also investigated whether the errors in various diagnostic tesis could be viewed as independent events. 


lymphangiogram?) ‘The data did not compel us to reject the independence assuraptions, but often we had too 


few data points to allow any definitive state bute MEAL tte Be ha he hog PE 
In summary, we did not forget that our Bayesian procedure for revising. staging probabilities was based on a 


strong independence assumption. Some tests of the assumptio n's validity were in order and were performed. 


Except in one case (sex vs. histology and symptoms), the tests indicated that the assumption was close enough 
to the truth that using it did not introduce serious ero in he calculations. In the case just mentioned, where 


the indepcadence assumptions failed! , we recalculated p pabilitie "A Jess direct method of testing our 


assumptions was the full-scale evaluation of the program as described in Chapter 5. This evaluation showed 
that our predictions of stage, even when many findings were used by applying Bayes’ Rule repeatedly, were 


reasonably accurate. This would not have occurred had our assumptions of independence proved false. 


We feel that our procedure, including the tests of independence, came as close as possible to avoiding the twin 
evils of biased predictions and predictions with very high variance, either of which would have doomed our 


efforts. 


1. One might wonder whether, when many tests of independence. are performed, chance alone would lead one of them to yield a 
negative result. In this situation, the rejection was sufficiently emphatic that we did nol think this was the case. 
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5. Testing the Bayesian Model for Predicting Pathological Stage 


In Chapter 4, we showed that the independence assumptions that underlie the Bayesian calculations of staging 
_ properties were consistent with the data used to calibrate our computer program. But, there.is another issue 
that must be addressed before the progr can be considered for general use. Can a model based largely on 
the experiences of patients from ‘age metropolitan teaching hoi reliably be used i in =e prognoses at 


other hospitals? 


There are several reasons one might fear this question should be answered in the negative. More than half the | 
patients in the database came from a well-known radiotherapy. referral center, which is perhaps more likely 
than other centers to attract patients with localized. disease, This circumstance raises the possibility that 
probabilistic predictions based on data dominated by these patients, might be overly optimistic for patients 
elsewhere. Furthermore, the data base. consists solely. of laparotomy-staged. patients. This could lead to 
underestimation of the prevalence of stage IV disease, for some patients in this stage could have had their 
extra-nodal involvement confirmed without laparotomy. Moreover, the predictions depend on the histologic 
subtypes reported for the patients Pathologists tend to differ somewhat in their assessments of histologic 
subtype (Jones et al, 1977); to the extent that patholo é differ from their counterparts eisciiete the 


“transferability” of results i is diminished. 


Finally, each of the prior and conditional probabilities used in the program has an uncertainty due to 
sampling error, an uncertainty which may be increased by the repeated use of Bayes Rule. In principle, we 
could estimate the uncertainty in our final staging probabilities asa function of the uncertainties in the 
numbers that determine it. However, the calculations would be very difficuit in practice, and ultimately of 


quite limited value if we could demonstrate that the program makes accurate predictions of stage. 


The considerations suggest the importance of testing the gencral predictive power of the decision analysis 


program. In this Section, we describe 2 separate empirical tests, one conducted in 1977 and the other in 1980, 
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Hust 


ee 


Both tests, as we will see, suggest that the program’s predictions are accurate to within an irreducible 


margin-of-error caused by usual chance fluctuations. 
5.1 The 1977 Test 


The test population we used consisted of 156 Hodgkins disease patients, 82 of whom were treated at Center A 
and 74 of whom were treated at the Tufts-New England Medical Center Hospi We thought it appropriate 
to consider the two groups of patients separately, so the Gaiecallig: but opposite patterns, if they arose, 
ined about histology, 


symptomatology, age, sex, presenting lymph nodes, spléen‘ and liver scan résults, ‘and ‘alkaline phosphatase 


would ‘not cancel one another out. For each ‘patient, itiforination ‘was’ ott: 


levels! . These data were fed into our program, which used them to spécify a probability distribution for the — 
patient’s stage under the Ann Atbor criteria. To assesd the accutacy of the program, we compared these 
probabilistic predictions with the patient’s actual stage as determined by taparotoniy. 


Neca 


An obvious question arises: how does one tell if a probabilistic prediction is accurate? Suppose, for example, 
the program predicts that patient A has a 60% chance of being i stage I or Mt and a 40% chance of being in 
stage III or IV. Even if the patient is found through laparotomy to be ia sage TIL, the prediction is not really 
wrong, for it had indicated considerable uncertainty about how far the diseaye had spread. But, while it is 
difficult to talk meaningfully about the accuracy of a given patient's prediction, it is somewhat susie to do so, 


given a large set of simifar patients. 


Suppose, for instance, that there are 50 paticnts for whom the program makes the 60%/40% prediction just 


mentioned. If these predictions are all accurate, one would expect that, except for random fluctuations, the 


eit 


"number of patients who actually emerge in stages I and I would be 50 X .6 = 30. Thus, if the number 


observed in the two carly stages is significantly greater than 30, this means the program was too pessimistic for 
1. Not all information was available for cvery patient. 
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these patients. If, on the other hand, the number is far below 30, the program was too optimistic. 


Our statistical test of the program’s accuracy for the two group of patthts'i based on the principles cited 
above. In performing the test, we combined stages I+II because, for clinical purposes, the two stages are 
virtually the same: We also combined aie Il = IV because, given the small test populations, we expected: 
so few stage IV patients that we will be unable to make meaningful ieee about the'stage IV eae 
For each patient in the test, we recorded that program’s estimate of "r", the probability the patient is in stage 
+L The probability that the patient was in stage III or IV is simply [1-r]. We then ordered the patients 


according to their r-values, from highest to lowest. 


According to the criteria discussed in Appendix II, we broke the ie into groups within which the r 
values are close together. Finally, we compared the expected number of stage I+Il patients in each group. 


with the actual number as learned from laparotomy:: We present some of ouf results in Tables 5.1-5.4: 


Table 5.1. Evaluating the TUFTS patients, using Histology, Symptoms and Sex — 


Group Number Average No. in Stages I+1l W-Statistic 

in probability 

Group in group Expacted Actual 

1 22 0.71 16.64 16 -0.26 
2 26 0.67 14.90 17 0.84 
3 26 0.3 7.72 8 0.12 

74 38.17 40 

Z = 0.79 


In the tables the W-statistic for any group is a measure of whether the discrepancy between expected and 
actual outcomes can reasonably be attributed to chance alone. The statistic has only a 5% chance of exceeding 


2 in absolute value if all predictions in the corresponding group are correct. Under the nut! hypothesis that all 
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Table 5.2. Evaluating the Center A patients, using Histology, Symptoms and Sex 


Group Number Average No. in Stages I+II W-Statistic 
in probability . 
Group _ in group Expected Actual 
1 16 0.7 11.24 12 6.41 
2 28 8.6 16.84 19 6.84 
3 21 0.44 9.26 8 -6.63 
4 22 0.31 6.73 8. - 8.66 
-Z 2 1,64 


Table 5.3. Evaluating the TUFTS Patients, using Histology, ‘Srmptoms, Sex, Age an Alkaline Phosphatase Levels 


Group Number Average No. in Stages: +H. Ww Statistic 
in probability 
Group in group Expected Actual 
1 23 0.74 16.91 16 -0.81 
2 a + a  §.61 an 8.62 °° Ww... 8.66 
3 14 0.5 7.06 & 6.25 
4 21 6.28 6.81. 6 ~@.41 
Z = 1.89 


Sex, Aus god Left-Neck Nodes 


Table 5.4. Evaluating the Center A Patients using Histology mS 
Group Number Average to. in Stages I+II W-Statistic 

in probability . oon 

’ Group in group- Expected Actual 
1 17 0.76 12.74 14 O.7 
2 15 0.67 10.1 8 -1.16 
3 18 0.57 10.28 13 ‘1.30 
4 16 0.47 7.08 6 -0.66 
5 22 0.32 7.03 6 -0.48 
Z = 3.60 


predictions in all groups are correct, the statistic Z obtained by summing ca squares of the W-values should 
follow a Chi-squared probability distribution, with as many siegitse ef freedons ‘a duseé sik gieank Thus at 
"the usual 5% level, one would reject the null hypothesis only if Z exceeds 7.81 when there are three groups, 
9.49 when there are 4 and 11.07 when there are 5. Tables 5.1 and 5.2 indicate that the projected distributions 


on stage, based on histology, symptoms and scx were accurate at both the New England Medical Center ~ 
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Hospital and Center A. Tables 5.3 and 5.4 suggest that the initial probabilities, modified in Bayesian fashion 
as additional findings become available, are also accurate, Revised predictions for the subset of patients who 
also had liver and spleen scans were as consistent with the ata as those summarized in the tables. Fhese 
tables imply that our concern that the predictions largely derived a a radiotherapy referral center would 


not be relevant elsewhere has not been realized. 


At this point the reader might be wondering why, if the initial predictions in Fables 5.1 and 5.2 were accurate, 
we bothered to revise them. The reason is that we need predictions that are not only accurate, but as helpful 
as possible. To a clinician deciding between treatments it is far more informative to ae! that the probability 
of a particular contingency is 0.99 rather than 0.50. Since including new findings in the predictions tend to 
polarize them towards 1 (certainty and zero (impossibility), the revised predictions are generally smuewiae 


less equivocal than the early ones. 


Once the Tufts and Center A patients were shown tobe ‘compatibfe with the original- database, it seemed 
appropriate to include their data in an expanded data base. The program that resulted from merging the data 
was based on 900 rather than 700 patients. Of course the very compatibility of the new patients with the 


original ones means that the merger brought only slight changes to the program. 


5.2 The 1986 Test 


In 1980, we decided * perform another test of the validity of the program outside the hospitals ‘arenes in its 
calibration. The reasons for this new test were that (i) the 97 data samples were too small to allow the 
consideration of stage IV predictions in their own right, and (ii) the ele inherent in the variable 
ieccement of histology may have been artificially reditced in the 1977 study because the test hospitals’ were 


similar in terms of histopathology. 
The 1980 test population consisted of all new pathologically-staged cases of Hodgkins discase arising in 100 
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ate ane eas te: Leen eee Ore 2 ectoed peed ot Ee Confidentiality of data was 
maintained, with both patients and hospitals indented by stady numbers ony. The original database was 
searched to insure that none of this test population was inchded in the existing database Clinical 
information about these patents came from 70 hospitals in the study area; in al, data on histology, sex, 
symptoms, age, and sites of lymphatic involvement was available for 301 pathologically-staged patients, 206 of 


whom had lymphangiogram tests. 


| The method for testing accuracy was similar to that used in 1977, except that we wanted to carn about the 
reliability of all three of the predictions of stage rather than just the two groupings used pieviously. The test 
procedure, whichis decribed in detail in Appendix I, involved choosing at random one ofthe thre 
predictions made for each patient. We called the stage to which che ies prediction for a oven patient 
referred, the “stage of interest". Thus, given 301 pationts, we were really atseming the accuracy of 100 
predictions about stage I+ H, 100 about HI and. 100 about IV. 


Tables 5.5 and 5.6 summarizing our test results appear below. The W-siatistcs can be interpreted the same 
way as those that appeared in tables 5.1-5.4. The hypothesis that all predictions are accurate is statistically 


unacceptable at the 5% level if Z is greater than 16.92 for 9 groups, and 18.31 for 10. 


Tables 5.5 and 5.6 make clear that the program’s predictions are accurate to within the level of chance: 
fluctuations. Thus, once again, we can have confidence in the wee pone assigned to various stages, 
and in the somevhat modified estimates tat are when addons infomation (general incdig 


‘ lymphangiogram results) is taken into account. 


. In summary, our tests in 1977 and 1980 offered strong evidence that the various potential problems in our 
Bayesian program are of little practical importance. The results inerease our confidence that the program can 


be useful in a wide variety of hospitals. 
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Table 5.5. 1980 Test of predictions based on symptoms, sex and histologic subtype _ 


Number of staged patients with this information available: 301 
Actual number overall in "stage of interest": 99 | 
Expected number overall in’ "stage of interest": 100.9 © 


Group ‘Number Expected ‘Actual « o WStatistic 

in in "stage in "stage 

Group of interest":. of interest" — 
1 37 | 24.94 28 1.21 
2 20 12.84 11 -0.70 
3 11 5.92 § -0.06 
4 11 §.47 3 -1.42 
5 14 6.73 9 1.08 
6 12 5.56 4 -1.08 
7 13 6.91 :) -0.76 
8 19 7.61 6 -0.50 
9 34 11.07 9 -0.78 
10 17 6.18 5 -0.27 
11 113 9.78 14 1.60 

Z = 10.39 


Table 5.6. 1980 Test of predictions using clinical findings and lymphangiogram results 
Number of staged patients with this information available: 206 
Actual number overall in "stage of interest": 63 

Expected number overall in “stage of interest": 66.46 | 


Group Number Expected Actual W-Statistic 


in in “stage in “stage 

Group of interest" of interest" 
1 28 22.80 24 0.73 
2 17 11.87 9 -1,44 
3 12 6.86 7 a 30.18 
4 11 5.04 4 -- *0.83 
5 14 6.05 6 0.06 
6 18 6.02 3 -1.02 
7 106 9.81. ...-.—s- All : 0,63 

Z = 4,66 
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6. Example of a Typical Patient Session 


We will demonstrate the program with a typical patient session. The patient is a 25 year old male who has. 
Gropp en vies Somers uae 1 He has no other genesylized symmptomscamd bas sought medical 
attention because of a sowly-increasing mass on the let side of his neck. Biopsy of this left-neck mass 
revealed Hodgkins dsease of the nodular sclerosis histologic subqype Investigations performed at this time 
include a chest X-ray.and a liver-spleen scan, both of which were normal. We will use this basic information 
to predict the likelihood of each of the sages of the disease and we tis knowledge to plan furtber 


investigations tailored t to treatment. 


First the program asks the three question which give information relating to the three interdependent findings 
from which the prior probabilities are estimated. In the dialogue between computer and user, the computer’s 


questions are in this typeface, the user’s responses in italic typeface. 


Is the patient A (asymptomatic) or B (symptomatic)? . B 


What is the histologic subtype of the patient? on “Nodular sclerosis 
What is the sex of the patient? tee a. Male 


The program then refers to its database to give the appropriate prior probabilities, Le. 


Stages I+II 0.34 


Stage III ~ “9.50 
Stage IV 0.16 


Having established ‘thes¢, further questions are asked to charatterize the patient.” ‘With each question the” 


probabilities are modified by Bayes Rule, using the conditional probabilities of finding given stage. 


In what age group is the patient? £2 to 39 years 


This is the correct answer for this patient, but by pressing "?” all passible answers the program will accept are 


displayed, i.e. 
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Unknown 

Younger than 12 years 

12 to 39 years 

Older than 39 years 
*Display data about fact... 
“What if..... 


If the response "*WHAT IF..." is selected, the program displays the.posterior, probabilities foreach possible 


answer. 

Younger than 12 years: I+II 0.21 III 0.52 Iv 0.27 
12 to 39 years: I#1I1 0.39 II11 0.61. IV 0.10 
Older than 39 years: ItII 0.20 III 0.48 IV 0.32 


The "*DISPLAY DATA ABOUT FACT" option displays the data known about the relationship between the 


patient’s age and the stage of the disease, and the conditional probabilities of age given stage, by which the 


prior probabilities are modified. 
Younger than 12 years I+II 0.03 III 0.06 IV 0.08 
12 to 39 years I+II 0.84 III 0.74 Iv 0.45 
Older than 39 years I+Il 0.14 III 0.23 IV 0.47. 


After one enters the appropriate age group for this patient -- 12 TO 39 YEARS - the prior probabilities are 


modified by Bayes Rule, using these conditional probabilities, in this case altering them in favor of early stage 


disease. 

Stages I+II 0.39 
Stage III 0.61 
Stage IV 0.10 


The user is then asked consecutively about left neck/right neck and mediastinal nodes, which make modest 


changes in the probabilities. Again the same explanatory features are available. 


Were there any nodes detected on the left 


supraclavicular or cervical areas? Yes 
Stages I+II 0.37 
Stage III 0.53 
Stage IV 0.10 


December 1981 -49- Hodgkins Decision Analysis 


Were there any nodes detected on the right 
supractavicular or cervical srees? No 


Is tvere medtestinal involvement? 5 pee No- 
Stages I+II 0.39 
Stage III 6.59 
Stage IV 0.07 


Next, information is sted separately about be Areen; scan and the liver scan. ‘Once again the: same 
. explanatory features ae avaiable. ‘ANtough these Iverspeea som results wee obtained simltancons, 


sqeige Beatie 7 TMG Td 

because fer erent inert, uestns tout res ar aed para. 
ous it ath fige. . wits Oh eRe os 

What is the dppearents of ta spleen on scan? “Normal . 


Normal scan: 

Enlarged ne 16 cm) . 
Fiji ting. defects: sa 
Both ite cane 1 


¥: 


Wit iret sn the probate ae modified: 


Stiaes I*IlI 0.48 
Stage III 0.48 
Stage IV (iH 
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Was the uptake pattern of liver scan. normal or abaormal? 


Possible answers: 

Normal 

Abnormal 

Unknown 

*What if... 

*Display data about fact... 


_ Normal 
Stages I+II 0.49 
Stage III 0.49 
Stage IV 0.03 


The false positive and negative rates for the liver/spleen scan are given in. Table 1.2. They have been 
combined with information from the database regarding the likelihood of an involved liver or spleen in the 
various stages to give conditional prebabilities for the the probability ofa particular scan result, given the 


pathologic stage. 


Having used this basic information to characterize the ati and obtained the above pote 

probabilities of stage, the program now enquires which, if any, of ihe more invasive tests Rave been 
performed. For any that have, probabilities are further modified according to rages Rule, using false positive 
and false negative rates that have been derived from the literature, again modified where necessary to link 


results of the tests directly to. pathologic stage as in the example shown-for Gallium scan in Section 2.2.2. 


The tests asked about at om stage include bone marrow » biopsy y (BMBX), percutaneous liver biopsy (LBX) or 
BeiaREORORy with guided liver biopsy (PTX), salon scan of abdominal sii (GALL) lymphangiogram 


(LAG), and staging laparotomy (LAP). 


The user is then asked whether any a these test procedures arccontaindiama. For each ae the u user. 


is prompted with list of the recognized contraindications /- dae: e.g. forlymphangiogcam: 
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Does the ctteae have any of the fortontng coatraindicat ions for 
Tynphang togran? 

Todtde hypersenstt tivity 

Respiratory impairment 

Active theombophteb tt ts 


Any procedure which is contraindicated at this stage will not be used in constructing manager 


Ficclaloa analvile i Row ueod Wy apputach Gs TES? vases TREAT Gccincls ca whothes be neceteny 
this. stage to perform some: or all of the more invasive: tests, at whether treatment.can be: given without more 
ado. The option of using combised medality therapy is elso offered. - The: tiser.is. rentinded that,: whee: 
allowance should be made for a second remission with chemotherapy if relapse occurs after single modality: 
therapy. If the user elects to use combined modality therapy it is added to those considered in the TREAT 
branches of the decision tree; Ne eee ae oak a oe a ee 
craments woul then nla cmbned moda erry (CM tal nodal aon (TM and MOPP 
As all the possible. branches. of the decision tee ane anscennd, their. utilities are: displayed in tebuler form, as 


below, oe 


First step Average | Avg morbidity ‘iva morbidity Dollar ‘cost 
jin plan. 6-year OFS of. testing. ef treatment. of: plan. - 
Total nodal 0.5966 0.8 1609. a600 
MOPP 0.6742 6.0 306g 1000 
CN 0.6788 0.8 4566 4200 
Laparotomy 0.7306 885 3729 10666 
Gull iue scan 0.7307 aS $0880. 
Lymphang logram 0.7298 966 3728 10666 
Bone marrow @..7307 a6e@ re "| Se | 
‘Liver biopsy @. 7366 881 3729 10929 


First, note that the plans are listed according to their first step. If this is a treatment, then there are, of course, 


no further steps. If the first step is a test, the utilities shown are calculated for the entire Plan, and include any 


subsequent tests and treatment. For example, for the plan beginning with bone marrow biopsy, the plan with 


the highest utility, the whole plan looks like this: 


I+ 


Combined modality 


Negative = => LAP 


We iV. 


p = 0.01 


This plan shows that when bone marrow biopsy is performed, there is a 0.03 chance of this test being positive; 
this chance is low because the probability of stage IV in this paticular patient was only'8.02. Since a positive 
bone marrow biopsy is conclusive proof of stage IV disease, MOP?P can then be given without any further 
testing. Ninety-nine percent of the time the bone marrow aie will prove negative. At this point the whole 
TEST versus TREAT decision is approached again. ane best sep next, with the probabilities revised by the 
negative bone marrow biopsy, i is to perform a laparotomy, with the chances of I+IIB disease being A9, and 
Til & IV being 0.51. Once laparotomy has been performed, stages are known for certain, and no further 


decision is necessary. 


At this point the user is given the option of displaying a number of other items including: 
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All plans - these are displayed as decision trees similar to that above 
One plan - any individual um can be displayed by designating its first step 


Summary of plans - very similar to the previous tabular ies except that plans are displayed 
ranked according to their DFS utility. 


Verbose version of one plan - a particular plan, specified by its first step, is described in 
words. 


Patient summary - all the clinical and test data known about the patient are displayed. 


Following this DISPLAY option, the-user is presented with the further options which make the program so 
flexible that its use can be adapted to a particular patient, or to the experience of any hospital. These options 


include: 


Change patient information - further specified as: 


Clinical information 
Test information 


Modify information about tests or treatments - further specified as: 


Prior probabilities ; 

False positive/negative rates for questions or tests 

Laparotomy as a perfect test - can assign a FN pate 

Mortalities for tests - the rates used are displayed together 
_.. with an aption to change them _ 

Morbidities for tests - see Table 32 

Morbidities for treatments - see Table 3.2, 

Survival data for treatmgnts - these can be changed according 

to institutional experience. 
Consider a different treatment — : 
Delete a treatment from consideration 


Reanalyze current patient - if any of the MODIFY options are selected 
one must use this option to recalculate plans, 


Supply your own plan - here the user is given the opportunity to supply 
his/her own plan for management, and compare it with those 
calculated by the computer. 


- We will try some of these options for our patient. Imagine that there has been some doubt about his exact 
histologic subtype, and the pathologist notes it has some of the features of the mixed cellularity subtype. We 


want to see what difference this will make to the probabilities of stage, and hence 10 the management plans. 
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What do you want to do? Change patient information 
What do you want to change? Clinical information 


The clinical information known about the patient will then be displayed, and the item to be changed 
designated. In this case, changing the histologic subtype from nodular sclerosis to mixed cellularity changes 
the probabilities from I+ II 0.49 : III 0.49 : IV 0.03 to I+II 0.37 : III 0.55 : IV 0.08, ie. it increases the 


probability of more advanced disease. 


Further, supposing you are not very happy about the patient’s lung function -- he is prone to asthmatic 


attacks, which you feel may affect the mortality risk for both the lymphangiogram and staging laparotomy. 


What do you want to do? Modify information about tests or 
treatments 

What do you want to modify? Mortality rates for tests 

Which test? Lymphangiogram 


The current mortality for lymphang iogram is 0.001. 
What would you like to change it to? 0.005 


Which test? Laparotomy 


The current mortality for laparotomy is 0.01. 
What would you like to change it to? 0.02 


Also you wish to consider as an alternative treatment for this patient, the "sandwich" form of combined 
modality therapy, in which 3 courses of MOPP are given, then a course of radiotherapy, then a second 3 
courses of radiotherapy. Since this option replaces the more conventional combined modality therapy, it is 
first advisable to delete combined modality therapy from consideration, before indicating you wish to 
consider a different treatment. You will be asked at this point for the name of the new treatment and for 


appropriate disease free survival and morbidity data. 


What would you like to do? Consider a different treatment. 
Type the name of this treatment “Sandwich” 
Five year disease free survival for "Sandwich" given I+IIB: 0.75 
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Five year disease free survival for "Sandwich" given III8: 0.4 


Five year disease free survival for "Sandwich" given: IVB: 0.35 
What is the expected morbidity associated with Sandwich _. 3500 
What would you like to do? Reanalyze current patient 


Now, the program again asks whether any invasive tests have been done already, or are contraindicated. It 
also asks whether combined modality therapy is indicated. Although standard ‘combined modality therapy 
was deleted from the program, it was replaced by the "sandwich" combined therapy so we need to answer . 


YES to this question so that salvage will be considered after the single modality therapies. _ 


The computer then calculates a new series of plans, which incorporate the changes just made, i.e. the different 


prior probabilities, the increased mortality rate for the lymphangiogram and laparotomy, and the new 


“sandwich” treatment. 
First step Average Avg morbtdfty: Avg. morbidity ‘Dollar cost 
in plan 5-year DFS of testing § ‘of? Rwateent:: of plan’ 
Total nodal © 0.6386 0.0 1600 3500 
MOPP 0.6610 0.0 3000 1000 
"Sandwich" 0.6819 0.0 st “= 3800 - 
Laparotomy 0.6938 8656 3186 ‘8628 
Gallium scan 0.6942 869 3185 8814 
Lymphangiogram 0.6909 614 | 3248 6048 
Marrow biopsy 0.6942 860 3186 8473 
Liver biopsy 0.6939 871 3186..,.. 8919 


The plan with the highest disease-free-survival is the one beginning with the gallium:scan: 
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1+ p=.50. 
= => Sandwich 


Negative p=.97 @ 


= =>LAP 
Negative p= £97 Itl+IV p=.50 
= => BMBX = =>MOPP lel p=.13 
re 8 = => Sandwich 
S Positive p = .03 
Gallium = => MOPP - egative p = .9¢ 
= => LAP © 
Negative ps.0) Hi+4V:p = 87 
ss > LBX == > MOPP 
Positive p = .31 O . |. Positive p= 01... 
= => BMBX = => MOPP 
Positive p = .05 
= => MOPP 


Altering these factors has fed to different diagnostic plans. Once’ again, further testing before selecting 
treatment gives better average disease-free survival values. However, there is now less emphasis on the tests, . 


laparotomy and lymphangiogram, which have been estimated to have increased mortality rates. — 


December 1981 - 57> Hodgkins Decision Analysis 


Aree es rte ee ged ee ey (SAL es woke FeO pamcarh edt 2 na Rit ease By Mek ae a a SOR ERE PS ERS A Re PTs gE ai EE ea Ter ST Rg 


7. Analysis: Immediate Laparotomy in Young Asymptomatic Patients 
We wished to use decision analysis to answer two related questions: 
1, Are there certain presentations of Hodgkins’ disease (HD) which have such a low probability of 


infradiaphragmatic disease that staging laparotomy is not justified? 


2. If negative test results cannot give sufficient certainty of early stage HD to-obviate. laparotomy, 
should the patient proceed directly to aia and not be subjected ae staging tests? 


The decision being evaluated is shown in Figure 7,1,:which is “g'Sithple TEST versus TREAT decision tree. 
The test branch for laparotomy, includes its peri-operative and late infective mortality of 1%, and the option 
to treat each stage with-its optimum treatment. Since th optimum reamet for stage IIIA is being debated, 
we will evaluate the éciica separately for the two diffetent treatments which are now being used for IIIA, 
namely total nodal irradiation (TND ang combines: otodélity therapy (CM). “Extended smantle radiotherapy 
(EM) is evaluated in the TREAT branes for these. young asymptomatic. patients, as this would be the 


treatment appropriate to their clinical stage. 


If the utility of immediate treatment is higher than that after laparotomy followed by stage-specific treatment, 
the tests are indicated since, if they prove to be negative, they may obyjate.laparotomy. If, on the other hand, . 
laparotomy has the higher utility even when all the tests are negative, then, one might proceed directly to . 


laparotomy, with the knowledge that testing can never provide epar 


zh certainty of early stage disease to allow 


immediate treatment. 


This particular analysis concerns asymptomatic patients in the age range 12-39 years, the age group with the 
highest chance of early stage disease. The probabilities of stage used to calculate the utilities are derived from 


the findings listed below - one criterion is taken from each category: 
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Fig, 7.1. 
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** When combined modality therapy is being evaluated as the optimal 


treatment for stage IIIA, MOPP salvage of relapse after the single 
modality therapy will be included for fair comparison. 


Histologic subtype - only nodular sclerosis (NS) and mixed cellularity (MC) are considered. 


Male or female 
Involvement of right neck nodes (RN +/RN-) 
Involvement of left neck nodes (LN +/LN-) 


Involvement of mediastinal nodes (MED +/MED-) 


There are 32 possible combinations of these findings; each sct of probabilities is further modified in 


accordance with results of all tests prior to laparotomy being negative. For simplicity we will consider only 
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eight of these combinations i.e. for each sex/histology combination we will show the calculated utility for the. 


presentations with: 


the most "FAVORABLE" clinical findings - LN-, RN-, MED+ 

the most "UNFAVORABLE" findings - LN+, RN+, MED- 
Table 7.1 shows the average five year disease-free-survival (5 yr DFS) calculated for each branch of the 
decision tree in Figure 7.1, when the treatment for stage IIIA after isparotomy is TNE 


a oe 


- Table 7.2 shows the average 5 yr DFS for each branch of the decision tree when combined modality is used 


Table 7.1. Sake 
5 year Disease Free Survival with Laparotomy followed — 
by Single Modality Therapy or with Immediate Extended Mangle Radiotherapy 
PATIENT LAP WITH’ | EXTENDED 
a TMI FOR _ MANTLE 
STAGE TfI © RADIOTHERAPY 
iviia 
NS FEMALE "FAVORABLE" 79.9% 78.9% 
(0.93,0.07,0.0) . 
NS FEMALE "UNFAVORABLE" "7a.6% 78.7% 
(0.85,0.16,0.00) se sei 
NS MALE "FAVORABLE" : 79.6% ~ 78.3% 
(0.91,0.08,0.00) 
NS MALE "UNFAVORABLE" 77.9% 74.0% 
(0.82, 0.17, 0.01) 
MC FEMALE "FAVORABLE" 78:9% 76.6% 
(0.88,0.12,0.01) 
MC FEMALE "UNFAVORABLE" 76.5%. 70.5% 
(0.75,0.22,0.03) San at | 
MC MALE "FAVORABLE"  -77.8% 74.0% 
(0.82,0.18, 0.01) 


MC MALE "UNFAVORABLE" 74.8% 66.5% 
(0.66,0.32,0.02) 
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for those patients found to be in stage IIIA at laparotomy. For comparison with immediate extended mantle 


radiotherapy, allowance is made for MOPP salvage of patients who relapse after any single modality therapy. 


Table 7.2. 
5 year DFS with Laparotomy followed by Single/Combined(IIIA) 
Modality Therapy or with Immediate Extended Mantle Radiotherapy 
PATIENT FINDINGS LAP WITH EXTENDED 
(PROBABILITIES) CM FOR MANTLE 
STAGE III RADIOTHERAPY 

NS FEMALE "FAVORABLE" 87.7% 86.7% 
(0.93,0.07,0.0) 

NS FEMALE "UNFAVORABLE" 87.1% 84.1% 
(0.85,0.15,0.00) 

NS MALE "FAVORABLE" . 87.5% 86.1% 
(0.91,0.08,0.00) 
NS MALE "UNFAVORABLE" ; 86.8% 82.8% 
(0.82, 0.17, 0.01) 

MC FEMALE "FAVORABLE" 87.2% 84.8% 
(0.88,0.12,0.01) 

MC FEMALE "UNFAVORABLE" 86.1% 79.8% 
(0.75,0.22,0.03) 

MC MALE "FAVORABLE" 86.8% 82.8% 
(0.82,0.18, 0.01) 

MC MALE "UNFAVORABLE" 85.5% 76.8% 


(0.66,0.32,0.02) 


For all the combinations of clinical findings considered, and with both single and combined modality 
treatment options, laparotomy is calculated to have the highest average utility. However, for some of the 
more favorable clinical presentations there is only a small difference in outcome between laparotomy and 
immediate treatment. Only when the probability of stage III disease exceeds 0.20, e.g. the 


"UNFAVORABLE" MC presentations, is the difference in average five year DFS between the TEST and 
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1! EAT options above 5%. Other factors than disease-free survival may influence the decision, for example 


‘Ae opportunity to perform oophoropexy in female patients. 


‘f one cannot obviate laparotomy even when all other staging tests are negative, we feel it is preferable to 
rroceed directly to laparotomy once the diagnosis of Hodgkins disease is made in this group of asymptomatic 
patients in the age group between 12 and 39 years. We particularly favor this course of action when certain 


unfavorable clinical findings give an appreciable chance of stage III disease. 
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8. Analysis: Laparotomy in Symptomatic Patients 


In using the Bayesian diagnosis program we found that certain cohorts of symptomatic patients with a positive 
lymphangiogram have a very low probability of early stage disease. This study was carried out to determine 
whether, amongst these cohorts, there were any for whom the risks of staging laparotomy (LAP) outweighed 
the benefits it conferred through knowing the stage accurately. This work was first performed and published 
using the 900 patient database (Rutherford et al, 1980). The present study uses the prior probabilities 


calculated from the 1200 patient database. 


For this study the probabilities of stage were calculated for cohorts of symptomatic patients, with a cohort 


being defined by one criterion from each of the following categories: 


(1) Nodular sclerosis (NS) or mixed cellularity (MC) histologic subtypes 
(2) Age 12-39 years, or older than 39 years 
(3) Male or female 
(4) Involvement/non-involvement left cervical lymph nodes. 
(5) Involvement/non-involvement right cervical lymph nodes. 
(6) Involvement/non-involvement mediastinal lymph nodes. 
All patients were assumed to have a positive lymphangiogram and a negative percutaneous bone marrow 


biopsy. 


There were thus 56 different combinations of findings, for which probabilities of stage were calculated, again 
using a simplified three stage version of the Ann Arbor criteria (Carbone et al, 1971), in which stages IB and 
IIB are combined. There were too few paticnts with the lymphocyte predominant and lymphocyte depleted 


subtypes for meaningful data in these categories. 
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8.1 Calculation of Decision Making Thresholds 


A decision-making threshold was calculated at which the result of treatment after laparotomy, taking into 
account the mortality of the operation, was equivalent to immediate combination chemotherapy treatment 
with nitrogen mustard, vincristine (Qncovin), procarbazine and prednisone (MOPP) (DeVita et al, 1980). 


Treatments planned after laparotomy were total nodal irradistion-(TN¥) for-auges IB- AND TIB, and MOPP 


chemotherapy for TIIB and TVB. Estimates of the probability Gf ave yer Ghéitée five survival (DFS) for a 
given stage with these treatments are shown in Table 8.1. These resid ait obtained from: data from the 


literature (Aisenberg et al, 1976, DeVita et al 1980, Goodman et ai 1977, Rosenberg et al, 1975). 


THI wopp 
Stage IB and IIB. 70 60° 
Stage IIIB 26 50 
Stage IVB o* 36 


* = estimate 


The probability of 5 year disease-free survival after laparotomy is calculated by adding the products of the 
probability of each stage, derived from the computer and the results of the best treatment for that stage; this is 
then corrected for the mortality of the operation (peri-operative and post-in fective), ie. . - 
COFStyy r+rr P(I+IL) + DFSwopp pry PC IIT) + OF Sygpp, ry. P(IV)] (1 - mortality) 
If we substitute the appropriate DFS values from Table 8.1 this expression becomes: 

[0.70 p(I+II) + 0.50 p(IIT) + 0.35 p(IV)] (1 - mortality) 
The probability of 5 year disease-free survival if immediate MOPP therapy is given is: 


DFSmopp,1+11 PCI+IT) + DFSwopp, riz PCTIT) + DFSwopp, ry PCIV) 
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Again, substituting the values from Table 8.1 this becomes: 


0.60 p(I+II) + 0.50 p(III) + 0.35 p(IV) 
P(IV) can be expressed as [1 - p(I+II) - p(IID]. The threshold between laparotomy and immediate MOPP 
therapy - found when the post-laparotomy disease-free survival is equal to that after immediate MOPP - can 
then be written as a linear equation. If, for example, one assumes a 1% total mortality from laparotomy, 
substitution of this in the laparotomy expression gives a linear equation for the threshold in terms of the 


probability of stage (I+ ID and III: 


9.65 p(I+II) - 0.15 p(III) = 0.35 
This threshold is displayed in a graph with the probability of stage (I+II) and stage III on abscissa amd 
ordinate respectively and a line drawn between 1.0 on each axis, to outline the area relevant to the study 


(Figure 8.1). 


Any patient cohort can be represented by a point on this graph in terms of the probability of stage I+- II and 


stage III. The nearer the origin of the triangle, the higher the probability of stage IV. 


The threshold between laparotomy and MOPP marks off an area where there is a low probability of stage IB 
and IIB. To the right of this threshold line is the area where laparotomy is the best course of action, despite its 


mortality. 


The cquation for the threshold between laparotomy and MOPP is affected by the estimated mortality for 
laparotomy. As the estimated mortality of laparotomy increases, this threshold shifts to the right, diminishing 
the area where laparotomy is the preferred course of action, and increasing the area where immediate MOPP 


should be given (Figure 8.2). 


The equation for the threshold if the laparotomy mottality is 2% is: 
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Fig. 8.1. : 
Decision thresholds between LAP and MOPP (Threshold i) and LAP and TNI 
(Threshold ii) when LAP mortality is estimated as 1%. 


MOPP 


P¢iil) 
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9.3 p(I+II) - 0.3 p(III) = 0.7 


and if laparotomy mortality is 3%, the threshold equation becomes:: 


8.95 p(ItII) - 0.45 p(III) = 1.06 
- If there is uncertainty in the results of therapy, the threshold equation reflects this, and the threshold line 
becomes blurred. For example, if our estimates for DFS are off by 0.03 in opposite directions, and the 


difference between the results of the two treatments is at a maximum, the threshold line between MOPP and 
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Fig. 8.2. The effect of increasing LAP mortality estimates on. decision thresholds. 
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Values for varying LAP mortality are indicated. 


laparotomy is shifted to the left. Conversely, when there is the least difference between results for each 
treatment, the threshold line is shifted to the right, making the value of laparotomy less. This "blurring" of 


the line is accentuated by increasing mortality of laparotomy €Figure 8:3). 


If laparotomy has a false negative rate the decision threshold is shifted in such a way as to diminish its value. 
False negative rates of laparotomy of 1% and 5% are considered. In this context “false negative" refers to true 
stage III patients who are incorrectly called stage (1+ II), and to stage IV patients mis-staged, with one quarter 


of the latter being called stage (+11), and the remainder being called stage IIT. 
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Fig. 83. ‘The effect on decision thresholds of 3% uncertalyty in trettment results 


1.0) Threshold (i) 


P(Il) 
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The dotted lines on either side of the solid lines bound the areas — 
‘included when 3% uncertainty in treatment. pesults ae nares into account. 

(LAP bide is estimated as 1%.) 

Analogous threshold lines can be drawn between laparotomy and immediate: TNI;. these mark off. triangular 
area where there is a very high probability of istages.IB. and‘ 1IB;’.We found, however,- that there were no 
cohorts of symptomatic patients - not even those with the most favorable combinations of findings and 
" negative lymhangiogram - whose probabilities fell within the area to the right of the laparotomy/TNI 
threshold, where immediate TNI would be the better option. This was so even when laparotomy mortality 


was increased to 3%, when uncertainty in therapy results was considered and when false negative rates were 
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considered for laparotomy. 


8.2 Results 


A point representing each of the fifty-six sets of probabilities of stage derived by the computer for 
symptomatic lymphangiogram- positive “patients” was plotted.on a graph. like those shown in figures 8.1-8.3 
to determine which decision was appropriate for each cohort: “These results are oresented in tabular form. 

Table 8.2 shows results when the only variable affecting the threshold was the mortality of the staging 
laparotomy. With laparotomy mortality estimated at 1%, six of the 56 cohorts had such a high probability of 
late stage disease that the point representing their probabilities was plotted in the area where. MOPP therapy 
could be given immediately. If the ciiGrtalltys of LAP’ was 2%, andthe threshold line moved to the right 
(Figure 8.2), the "immediate MOPP area” included afl mate MC ‘patients with positive lymphangiogram, 
together with several other cohorts which represented the less ivorable findings for the other categories. If 
the laparotomy mortality was estimated to be 3%, ali male patients with Bonitive iymphangiogram are plotted 


in the “immediate MOPP area", together with more than half the ccs of: female pale considered. 


Tables 8.2-8.5 show that the numbe of cohorts of patients for whom immediate MOPP was justified increased 
considerably, when some of the variables influencing the decision, namely the false negative rate for 
laparotomy and the uncertainty of the treatment results, were considered. Fhe uncertainty in treatment 
results influenced the decision most, but considering false negative and false positive rates for laparotomy also 


increased the number of cohorts for whom immediate MOPP. was indicated. 
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Table 8.2. 
SYMPTOMATIC ADULT® PATIENTS WITH POSITIVE LYMPHANGIOGRAM FOR IMMEDIATE MOPP 


LAP. HIST. SEX ; PATTERN OF NODAL INVOLVEMENT 
MORT TYPE N-N- N+N- N+N+ N+N- N+N+ 
M+ M+ M+ N- M- 
1% MC M - >39 >39 >39 >39 
2% NS M >39 >39 >39 ALL ALL 
NS F - - - >39 >39 
MC M ALL ALL ALL ALL ALL 
MC F ~ - >39 >39 >39 
3% NS M ALL ALL ALL ALL ALL 
NS F - >39 >39 >39 ALL 
MC M ALL ALL ALL ALL ALL 
MC F >39 >39 >39 ALL ALL 


*Adult - older than 11 years 
>39 - refers to patients older than 39 years 


ul 


Mediastinum N.= Cervical lymph nodes 


+ 


involved with Hodgkins disease 
- = not involved with Hodgkins disease 
i.e. N+N+ = Bilateral cervical node involvement 


N-N- = Cervical nodes not involved 
N-N+ = Unilateral cervical node involvement - right- or left-sided. 


8.3 Discussion 


From basic clinical information and the results of the lymphangiogram we have been able to predict the 
likelihood of the various pathologic stages in symptomatic HD, and to compare the value of immediate 
chemotherapy with that of laparotomy in a group of patients who have a low probability of early stage disease. 
With the simple decision-making techniques used here it is possible to analyze the effect of three important 
variables on the decision to perform laparotomy - its mortality for a given patient, the validity of the results of 
laparotomy and the reliability of treatment results on which management decisions depend. Even with the 


conservative figures we have used for these variables, this approach could save an appreciable number of 
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Table 8.3. 
SYMPTOMATIC ADULT* PATIENTS WITH POSITIVE LYMPHANGIOGRAM FOR IMMEDIATE MOPP 
IF THERE IS 3% UNCERTAINTY IN TREATMENT RESULTS 


LAP. HIST. SEX PATTERN OF NODAL INVOLVEMENT 


MORT TYPE N-N-  -N+N-  oNEN+)ONSN~ NEN 
Me oMt Me ON M- 
1% NS M 2 >39 >39 ~—ALL ALL 
NS F a ~ = »>39 >39 
MC M ALL ALL ALE-° ALL ALL 
MC F : oy >39 >39° «#39 
2% ALL 
3% ALL 


For key to table, see Table 8.2 


Table 8.4. : ae: 
SYMPTOMATIC ADULT*® PATIENTS WITH. POSITIVE. LYMPHANGIOGRAM 
FOR WHOM IMMEDIATE MOPP IS. THE BETTEB::OPTION 
WHEN LAPAROTOMY IS CONSIDERED TO HAVE A 1% FALSE NEGATIVE RATE 


LAP. HIST. SEX PATTERN OF NODAL INVOLVEMENT 
MORT ‘TYPE N-N~ NN] ONSNY CRN NEN 
M+ M+ N+ M- 0 N- 
1% NS M 5 P = -  —-»39 
MC M >39 >39 >39 >39 
2% NS M >39 >39 ALL ALL ALL 
NS F = 2 >39 39. ~=—«» 39 
MC M ALL ALL ALL ALL ALL 
MC F a >39 >39 >39 >39 
3% NS M ALL ALL ALL ALL ALL 
NS F >39 >39 >39 ALL ALL 
MC M ALL ALL ALL ALL ALL 
MC F >39 >39 ALL ALL ALL 


CNN ele ee eee ee ee ee ee er ee ee 


For key to table, see Table 8.2 


patients the rigors, risks and expense of a LAP, and of the other tests frequently performed in routine staging. 


In certain patients the risk of laparotomy mortality may well be higher than 1%. Immediate postoperative 


mortality for 2345 patients from 27 different series was 0.5% (Table 1.3); none of these series reported 
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Table 8.5. 
SYMPTOMATIC -ADULT*®: PATIENTS. WITH POSITIVE LYMPHANGIOGRAM — 
FOR. WHOM: IMMEDIATE: MOPP. IS THEUBETTER OPRFEON 
WHEN LAPAROTOMY. IS CONSIDERED TO HAVE A 5X FALSE NEGATIVE RATE 


LAP. HIST. SEX PATTERN: OF NODAL INVOLVEMENT 


MORT TYPE N-N- N+N~-- N+N+ = —=N+N-- N+N+ 
M+ Me. Me. M- - M- 
1% NS M nae >39 . >39 >39 :* ALL 
MC M >39. >39 ALL - ALL ALL 
MC F = - - m => +> ->99- 
Re Rios rim eee Senin oe nine ani Remain RS see emits 
2% NS M ALL ALL ALL ALE ALL 
NS F = >39 >39 >39 ALL 
MC M ALL ALL ALL ALL ALL 
MC F >39 >39 — ALL ALL ALL 
laa aaah ata adleaaka esta acteatentat atachatac aha ahaa etated ale teetetetatate Sr tomers ses cee 
3% NS M ALL ALL ALL ALL ALL 
NS F ... .>39 239... _ ALL ALL ALL .. 
MC / ALL ALL ALL ALL ALL 
MC F >39 ALL ALL ALL ALL 


oe ew me ee oe ee ow tabby Bee oe a thc np wn ld wd mi i io at 


For key to table see Vietees ‘for: ‘Tabie 8. ia AAT 


age-specific rates, but there my be wide aviation, with muck ler ras in young patients pie higher rates: 
~ in older patients, or those with cilicc complicating: factors: in addition to » post-operative mortality, there may 
also be a delayed mortatity due to the fulntifant sepsis’ “known to occur in a small préportion of 
splenectomized patients; this usually occurs within 2 years of Splenectomy ‘(Krivit, 1977). Wes Desser and 
Ultmann (1973) collated results from 1190 patents splenectomized as part of staging laparotomy, they found 
that late death from overuéwtiming sepsis, unrelated to treatment induct eucopeniair other precipitating: 
cause, occurred in 0. 5%, ‘Other series epore higher rates of Fpost-spleagctomy sepsis; for example Singer 
(1973), reviewing 2975 splenectomized patients, found the overall Jate death rate from infection was 2.52%; ‘ 
even patients who had splenectomy incidental to another operation had an_appreciable mortality from this 
- complication (0:86%). Furthermore, suaceptibitity 10 infection after splenedtiity may be exacerbated by 
treament-induced marrow suppression, (Schimpff & O'Connell, 1977), and by the defects in cellular 


immunity frequently noted in patients with HD (Desforges et al, 1979). 
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In this study, the mortality of laparotomy is considered to be its only disadvantage, and one must remember 
its other liabilities. In addition to mortality, 6.7% of the 2345 patients in Table 1.3. had serious complications 
of the staging laparotomy; these included such major post-operative problems as subphrenic abscess, 


pulmonary embolism or intestinal obstruction requiring reoperation. 


Uncertainty about results of treatment should influence therapeutic decisions. For example there is no firm 
evidence that any treatment has a particular advantage for'stages 1B and HB HD; recent trials (Goodman et al 
1977, Rosenberg et al 1978) have not demonstrated a statistieafty significant difference between TNI and 
combined modality treatment. All reported series of pathologically-staged 1B patients contain less than 25 
patients in any particular treatment group (Aisenberg. & Qazi, 1976, Fazekas et al 1975, Fuller & 
Madoc-Jones, 1977, Goodman et al 1977, Rosenberg et ‘al 1978, Stoffel:& Cox, 1977). In these small groups 
results for TNI vary between 79% and 48% 5 year disease free survival. For a series of 25 patients, 95% 
confidence limits for a disease free survival of 79% range between 90% and 60%, and when discase free 
survival is 48% they range between 72% and 27% (Greenwood & Hartley, 1962). The uncertainty rate we have 
used (3%) in these calculations is, therefore, a very conservative estimate for uncertainty in TNI results. Even 
this modest uncertainty, however, made a considerable difference to the numbers of patient cohorts for whom 
immediate treatment was appropriate. The MOPP chemotherapy regime for treating disseminated disease 
has large series (Aisenberg & Qazi, 1976, DeVita et al, 1980) reported with long-term follow-up and much 
narrower confidence limits; there is, however, little information on its use in localized disease. Some data on 
the use of MOPP for early stages has come from Uganda (Olweny et al, 1978), but this patient group is 
scarcely comparable, since their patients were predominantly young boys, who had minimal clinical staging; 
moreover, separate data for A and B patients is not gies Our figures for MOPP for stage IB and IIB are 


estimates, therefore, and subject to uncertainty, as are the estimates for TNI for disseminated discase. 


A further complicating factor in the decision is the accuracy of laparotomy. Although physicians use the 


results of this operation as the final answer, there is likely to be a false negative rate, due to sampling error. 
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This sampling error is difficult.to quantitate; Cotman et al-(1977) showed. that 18 of 88 Hodgkins disease 
patients still had lymphographically-abnormeal nodes after LAP. A 1% false negative rate secms realistic for a 
surgeon experienced in this type of surgery; a 5% false negative rate may be.appropriate for a surgeon not 


experienced in staging laparotomy. 


In spite of these uncertainties in risks and outcome, decisions regarding staging and subsequent management — 
must be made, Tables 8.2-8.5 give guidelines for patients who may proceed directly to treatment with MOPP, 
with provision to include. the possible. variables: The same. principles of threshold analysis can be. applied: 
even if different treatments are used. Probabilities of pathological stage for a given. patient can. be obtained 
from Appendix 1; Bayes Rule can be-used to incorporate the-results of any test, using false positive and false 
negative rates appropriate to the institution ia which they have beer performed. -New decision thresholds can 


be derived by substituting the new treatment results in the threshold-equations. 
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9. Problems in Applying Decision Analysis to Hodgkins Disease 


There are some limitations in the Hodgkins disease decision analysis program which may restrict its 


usefulness. 


Utilities have posed the major problem Den we cats to evalaais our plans in terms of five year disease- free 
survival (5 yr : DES) we have had difficulty i in finding ag series sis pains for enon disease- -free survival 
- data is available. All too often the series are mull with aakiely few cateits followed through to the five 
year stage, so that the data has considerable uncertainty. For some stages and treatments there is no 


disease-free survival information available, for example for chemotherapy in early stage disease or for 


radiotherapy in advanced disease. These values, therefore, have had to be estimated. 


A further problem with the utilities of plans is that there is no reasonable way to "trade-off" or compare 
morbidity data with disease-free survival data. In the program these ee therefore, simply quantitated and 
expressed separately. Even comparing the different types of mortality is a problem. In our plans the "cost" of 
a post-operative death from laparotomy is balanced against survival disease-free at five years, where the latter 
is aed to mean cure. It seems certain, however, that most patients would not regard immediate death and 


relapse at 5 years as having equal “cost”. 


The program is also not useful in those cases with unusual presentations. Since it is dependent on data to 
calculate both the prior and the conditional probabilities of stage, unusual presentations are poorly 
represented and we have little confidence in probabilities calculated from these data because of sampling 
error, The management of such presentations as bilateral axillary node involvement alone, or of inguinal or 
other isolated infra-diaphragmatic involvement is not clear cut. For this reason these are just the type of 
patient for whom the physician would like some guidance about.the likely aptead of disease, yet because of 
the relative rarity of such cases our program does not have sufficient data to give reliable probabilities of 


stage. The same problem arises with patients with the less common histological subtypes - lymphocyte 
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depleted and lymphocyte predominant. 


There is also a problem with histologic interpretation. - Qur prior, probabilities of stage are quite strongly 
influenced by the histological subtype of the HD. There is, however, considerable variation amongst. . 
pathologists in their reporting of the HD neesy win ihs we sean cession (Jones et al, 1977). 
Since our testing of the database showed that the program made accurate predictions of sage, we have 


concluded that this uncertainty in assigning histologic subtypes is not oti major importance to our program. 
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10. The Computer Implementation 


The goal of this chapter is to reveal enough of the workings of the Hodgkins disease program to allow an 
interested programmer to reproduce it. The following topics will be addressed: 

1. _ representations of knowledge about ota daa disease, amet orale clinical Findings, tests, 

and treatments, and probabilities _- 5 

2. data structures for maintaining a description of a patient 

3. data structures for representing decision trees 

4. calculating probabilities of stage 

5. growing decision trees and making decisions 

6. subsidiary programs for displaying and modifying program data 


7. interactive features to deal with the twin problems of low band-width data transmission and 
use of typewriter input by non-expert typists. a 


8. additional features: threshold analysis, salvage analysis 
10.1 Introduction 


The Hodgkins disease computer system consists of two independent subsystems: a computer program for 
performing decision analysis in the management of patients with Hodgkins, and a database system for 
managing information about 2000 individual Hodgkins disease patients. Both subsystems are large, 
interactive Lisp programs devoted to ease and rapidity of me The persion of these programs from the 
user’s point of view has been discussed in other sections. ‘This section deals with the details of the 


implementation of the:decision analysis subsystem. 
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10.2 Maclisp as an Implementation Language 


Several features of LISP, and of the Maclisp dialect in particular, facilitated-the. development of the Hodgkins 
disease program. LISP provides: data structures appropriate to representing complex structures such as 
decision trees and canoer aos namely shia dhcp Maclisp, eeiee with the text editor Emacs, 
proves: a remarkable interactive programming eaviroaiient rege excellent debugging facilities: aThe LISP 
language, augmented by a macro package for iteration, provises costeol structures appropriate to creating and 
manipulating complex, repetitive, and recursive structures: ‘The: flexible. input and output capabilities of 


Maclisp allow the easy development of dialog between computer and user. : 


10.3 Data Structures 


The Hodgkins disease decision analysis program operates. en thee types of information: knowledge about 
Hodgkins disease, descriptions of the patient under consideration, end: decision trees. 


10.3.1 Knowledge about Hodgkins Disease 


knowledge of Hodgkins disease cennists of: 


1, identifiers for important aspects of the disease, inchuding symptoms, tests, and treatments (e.g. 
an important: aspect of a particular patient's Hodgkina disease isis -‘symptamatology” -- 
whether the pain is aan (A) or aor @) 


2 ‘auitiative relationships between these pies of the ecu tek ei program contains 
information denoting the fact that symptomatolage,- tan inppartantctiieiceh fading is: ee 
Hodgkins disease); and 


3. quantitative (probabilistic) relationships between aspects of the disease (ce. the probability of 
positive Ieft neck nodes in a symptomatic stage HI patient is 0.75). 


This knowledge is stored partly as the valucs of free (or global) variables in the Lisp environment, and partly 


as property lists. Free variables are uscd mainly to give names to useful lists of items, such as the list of 
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clinical findings or the list of tests. Examples of such free variables, and their default values, are shown below. 


The names of the variables are on the left, their values, in conventional LISP list notation, are on the right. _ 


_ Variable Value 


*prior-findings* ‘(histologic-subtype symptomatology sex) 
*conditional-clinical-findings® ‘(age left-neck right-neck mediastinum 
, splteen-size liver-scan) 
*clinical-findings® (append *prior-f indings* 

*conditional-clinical-findings*) 
*tests® "(galt lag bmbx bx} -— 
*all-findings* (append *clinical-findings* *tests*) 
*treatments* '(total-nodal mopp extended-mantie 

comb ined-modal ity) 

*stages* ‘(it+ii fii iv) 


These lists may be modified under user command. For example, if the user wants to have the program 
consider a treatment different from the four included in *treatments*, he invokes a request which runs a 


program which adds the name of the new treatment to the list *treatments®. 


Property lists are used to maintain detailed information about each of the findings, tests, and so on. Fora 
clinical finding, this information includes the list of values of the finding (e.g. SEX. may be one of (MALE 
FEMALE )), plus directions to the program about how to ask about the sex of the. patient and how to display it 
to the user. Such information is stored in a disk file, from which it is loaded when a new version of the 
Hodgkins program is constructed. For example, the information about SEX appears in a text file as the 
s-expression below: 
(*load sex . 

(kind clinical-specific) 

(results (male female)) 

(inquire (|What is the sex of the patient?|))) 
This expression is evaluated, to give SEX the property list shown-below. (The "INQUIRE" and."KEYLST" 


properties provide the program with information for asking the wser questions about the sex of the given 


patient.) 
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(RESULTS (MALE FEMALE) 


KIND CLINICAL~SPECIFIC 
INQUIRE | (|What is the sex of the patient?|) 
KEYLST | fo (MALE NIL KEYSTRING [Mate] RETURN ° MALE) 


(FEMALE NIL KEYSTRING |Female| RETURN 'FEMALE) 

(UNKNOWN NIL KEYSTRING [Unknown| RETURN ‘UNKNOWN) 

(| *WHAT IF ..4..?¥NIL KEYSTRING |*What if ... 7}. 

RETURN 'WHAT-IF) 
(| “FINDINGS: SQ: FARI «NIL KEYSTRING |[*Findings -so far| 
RETURN “END INGS-SO- FAR) 
(| *DISPLAY. DATA. Wik KEYSTRING |*Display datal 
RETURM DLSPLAY-DATA) )) 

For a test, the detailed information consis of the ene of the test, the cost of the test in terms of mortatity, 
morbidity, and money, Seanad the: contlitionat: probabifities aia test results and stage. For a Weinert: it 


includes numerical values for survival values, morbidity, and dollar cost. 


10.4 Patient Description 


Throughout a decision analysis session, the program maintains a saints of the patient, which is 
findings (e.g. SEX=MALE), for the known results of tests ey a out, and for the current estimates 
of stage probabilities. Once the decision analysis has been carried out, the set of recommended diagnostic . 
plans i is added to the patient description. For example, the patient described in the first paragraph of the last 


chapter would be represented internally as the property list peigwe” 


(PROBABILITIES. (0.486 0.485 0.029) 
A-OR-B B 

HISTOLOGIC-SUBTYPE NS 

SEX MALE 

AGE 11KAGEC400 
LEFT-NECK POSIT IVE“LEFT-NECK 

RIGHT -NECK NEGATIVE-RIGHT-NECK 
MEDIASTINUM -' WEGATIVE-MEDTAST HOON 
SPLEEN-SIZE NORMAL-SIZE-NO-DEFECTS 
LIVER-SCAN. _ NORMAL-LIVER- SCAM): ae 
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10.5 Probability Calculations 


The program uses Bayes Rule to calculate stage probabilities from the patient description. For reasons 
discussed elsewhere in this article, the three findings SEX, HISTOLOGIC~ SUBTYPE; and SYMPTOMATOLOGY - 
were selected as the basis on which to estimate the prior probabilities. Once the program has asked for the 


values of each of these findings, it looks up the initial probabilities in a table indexed by the findings. 


The program then asks for the other clinical findings and, using Bayes Rule, updates the probability estimates 
after each answer. Assumptions of conditional independence were made where intuitively reasonable and not 
contradicted by statistical data. Dependence was incorporated in two ways: by clumping dependent findings 
and by explicitly conditioning certain conditional probabititiés on the values of other findings. Clumping was 
used in creating the group of findings for initial probabilities. Explicit conditioning was used to deal with the 
dependence of several findings on the symptomatology of the patient. Further, ied, of the Adina were 
thought to be tests of more fundamental findings such as liver'or spleen’ involvement:‘In these cases, a two 


stage version of Bayes Rule was used: as described in Section 2.1. 


Our implementation of Bayes Rule in Maclisp calculates the list of posterior probabilities from the 
conditionals and the list of priors, following the mathematical statement of Bayes Rule, shown below for the 


probability of stage I+ II given positive right neck nodes (+ RN): i bs Dandie, «a ate Slay goes 


pas 


oa wages ate S 


This calculation is carried out by the function Bayes, whose definition in Lisp appears below: - 


PE aE “ee 
(defun Bayes (finding prior) ; 
(unitize ; normalize the list 
{for prob in prior as stage in ‘(I+II III IV) 


collect 
(times (Conditional-Probability-of finding stage) prob))))) 
;of the products of priors and conditionats 


For example, if the prior probabilities are '(..3..3 .4) corresponding. 10. stages '(14]1 TIT I v) ina B. 
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patient with a certain set of clinical findings, then the list of posterior probabilities given that the patient has a 


positive right neck node is (bayes '+RN'(.3 .3 .4)). 
The calculation of this quantity proceeds as follows: 


l. finding is bound to '+RN 
2. prior is bound to '(.3 .3 .4) 
3. Foreach prob in prior corresponding to the stages, the following product is calculated: 
P(+RN] stage) * prob | 
The list of these products is constructed, and then normalized to have sam 1.0 by the function 


unitize. This is exactly equivalent to dividing by the usual denominator in the expression 
for Bayes Rule, since.that denominator is just the sum of the uanormalized products. - 


- 10.6 Decision Trees and Diagnestic Plans 


Once all of the known information about the patient has been used to calculate the. stage probabilities, the 
program constructs diagnostic plans for the patient from the available tests and treatments. A diagnostic plan 


is represented by a recursive list structure according to the following BNF-like syntax: 


Plan := (Utility-Summary Utility-Data Branches) 

Utility-Summary := (Utility . Action) 
"Utility. is the numeric expected utility of the plan 

Util ity-Data is a breakdown of utility by. attributes! | | i 
Action := Treatment | Test | 


Treatment := THI | MOPP | EM 


1. Though space in the structure has been left for this item, the current version of the program docs not use multiple utility attributes 
except when summarizing an entire plan.al top-level. The space is filled with the atom ‘utility-data-place-holder. 
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Test := BMBX | LAG | LBX | GAL | LAP 
Branches := (Branch) | (Branch . Branches) 
Branch := (Test-Result Probability-of-Result Posteriors Plan) 


Test-Result is a result of sede test. Eg. if test is BMBX, then Test-Result 
may be either +BMBX or ~BMBX.. bart 


Probability-of-Result is a numerical value representing the probability that the 
result will occur, given the prior probabilities at that point in the tree. 


Posteriors is the list of probabilities after the result of the test has been taken into 
account 


Plan (second appearance) is the plan which is optimal if the given test result occurs 


For example, the plan shown below is represented internally as the list structure following it. 


Plan 1: The following diagnostic plan has an estimated DFS of 0.5873 for an 
A patient with probabilities: I+II 0.0337 III 0.3716 IV 0.6946 


/ 0.04 I+II EM 
/0.89 - LAP | 0.42 III TWI 
| \ 0.654 IV MOPP 

BMBX | 
| 
\ 0.11 + MOPP 


Internal representation: 


((0.6872 . BMBX) 
utility-data-place-holder 
((-BMBX 0.8886 
(0.0379 0.4182 0.5438) 
((0.5920 . LAP) 
utility-data-place-holder 
((PATH STAGE-I+II 0.0380 
(1.0 0.0 0.0) 
((0.82 . EXTENDED-NANTLE))) 
(PATH-STAGE-III 0.4183 
(0.0 1.0 0.0) 
((0.64 . TOTAL-NODAL))) 
(PATH- STAGE-IV 0.6438 
(0.0 0.0 1.0) 
((0.65 . MOPP)))) 
(+BMBX 0.1116 (0.0 0.0 1.0) ((0.55 . MOPP))))) 


The plans are created by exhaustively tracing a large decision tree of possibilities using a computer 


2. Sce previous footnote. 
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implementation of “averaging out and folding back". ‘The initiator of tree analysis is the. function 
Expand-Top-Level- Choice hogs, which > sprouts subtrees beginning ‘with each ‘of the possible | 
treatments (listed in *treatments®), the special test LAP, and other tests not already done o or otherwise 


contraindicated (tests- left), and evaluates the plan optically suppleisiby the user 


(defun Expand-Top-Level-Choice-Node (tests- left prtor Tep- done?) 
(append ; 
; Treatment plans 
(for treatment in *treatments* 
collect 
(ncons (cons (Treatment - -E-U treatment prior) treatment))) 
;Lap plan 
(ncons (Lap- Plan prior)) 
: Test plans 
(for tests on tests-left as test = (car. tests) 
"collect (Expand- Chance-Wode test. prior (car tests))) | 
; User plan” 
(and (get ‘controller ‘user-plan- form). 
(ncons (Instantiate-Plan -plan- form prior))))). 


Expand-Chance-Node is called by the top-level node expatider to grow ‘the subtree beginning with a 
particular test. It constructs branches for each of the possible results of the tests, expands each branch, and 
calculates the expected utility for the subtree. It returns to the calling program the expected utility and the 


structure of the resulting subtree. 


(defun Expand-Chance-Node (test prior tests- left) 
(for res in (test-results test) 
bind (cond-prob 0.0) posterior branch (wenttey 0. 0) 
(new-tests (cdr tests- left)? 
collect tied 
(setq cond-prob (one fot -rtabab tity -of res ephtoky. 
posterior (Bayes ré$ prior)” see asia of 
branch (if tee test ‘lap) “i - 
(Choose-Bbst«TFekuhei powkerioe)” 
 (EXpaid-Choice-Nnbde’ pésterior new-tests) ) 
utility (plus utility {tings Cond prob ‘@éaar branch)))) 
(list res cond-prob posterié! branéh 
returning (list (cons” (AgcoudtSrorsAaFedT ity, test utility) test) 
‘ut ility-dita-pracdsnotder’ . 
result))) 


Expand-Choice-Node constructs a subtree for each remaining test, beginning with that test. It evaluates 


cach subtree, and returns to the calling program the subtree with the highest expected utilily. 
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(defun Expand-Choice-Node (prior tests-left) 
(if (null tests-left) 
(Treatment-vs-Lap prior) 
(for tests on tests-left as test = (car tests) 
bind (best-treatment (Treatment-vs-Lap prior) ) 
choose-best (Expand-Chance-Node test prior tests-left) 
according-to ‘(lambda (x y} (TeSsp (taar'x) (caar y))) 
returning (if (lessp (caar result) (caar best-treatment)) 
best-treatment . 
result)))) 
(defun Treatment-vs-Lap (prior) 
(Jet ((lap-plan (Lap-Plan prior)) 
(best-treatment-plan (Choose-Best-Treatment prior))) 
(if (greaterp (:utility Jap-plan) (: utility best- treatment- plan)) 


Jap-plan — 
best-treatment-plan))) 


(defun Choose-Best-Treatment (prior) 
(for treatment in *treatments* 
choose-best (cons (Treatment-E-U treatment prior) treatment) 
according-to ‘(lambda (pi p2) (lessp (car P1) (car p2))) 
returning (ncons result))) 


For an example of a program trace showing these programs being run to create a diagnostic plan, see 


Appendix 4. 


10.7 Interactive Features 


We paid particular attention to the interactive features of the program for two reasons. First, our "customer" 
is typically a doctor with little expertise in typing, so we want to minimize required input to the program. 
Secondly, access to the timeshared computer which runs the Hodgkins program is via slow-speed (30 or 120 


char/sec) dial-up lines, hence it is important for output to be efficient as well as informative. 
10.7.1 Input 


To minimize typing without using graphical devices, we use contmand completion and tabular data entry. The 
former capability is used when asking the user a menu-selection question, a question with a fixed set of 


possible answers. The user typically has to type only one or two characters to uniquely determine a response 
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-- the program supplies the remaining characters, A simplified version of INTERLISP’s ASKUSER package 


was used to implement this feature. 


Tabular data entry is invoked when the user is required to supply. ormodify a table of numbers. For example, 


if the user wants to change the survival values for Hodgkins disease treatments, the computer displays the 


values as shown below: 

. «Y+TIA I+1IB TIIA. =6pYTB. CSA IVB 
TOTAL-NODAL 81 iy ee “840 26.35, et 05 
MOPP a 6 62. 8 BE 35 
EXTENDED-MANTLE .82 .62 a. on “16° 0 0 
CM .87 .87 86 66  .66. — 36 


This feature allows the user to "edit" a table of numbers using a “real-time” efit ‘The user moves a pointer 
or cursor up or down or across the table by means of. single shuractes commands, and. then changes the 


number at the pointer by typing in the new value. 


For example, suppose the user wants to change the 5-year DFS figure for MOPP in IV A disease from 0.55 to 
0.57. When the table is presented to the user, the cursor is in the upper left hand corner. Then the user types 
"D" to move the cursor to the second row and "R" 4 times to move the cursor to the fifth cahaini ‘Then i 
simply types the new number over the old one.. The new entry becomes permanent when .the user types any. 


character that is not a digit, "+", "=", or:".". 
10.7.2 Output: Display of Diagnostic Plans 


The primary output of the program is the diagnostic plan for the patient. In order to achieve the multiple 
goals of ease of comprehension, efficiency of display, and completeness of information, we employ two 


. distinct representations of diagnostic plans: branching tree and. outline format. 


The branching tree format has been acecniell in figures throughout the p paper, e.g. in section 11.6. The plan 


‘in that section beginning with BMBX is displayed in outline foitoia as follows: 
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Perform BMBX 
1.1 -BMBX with P = 0.89. peerersors = 0.04 0. 42 0.64 . Then: 
Perform LAP 
1.1.1 PATH-STAGE-I+II with p = 0. 04. Postenione = 1,00 0.00 0.00. 
Then: Perform EXTENDED-MANTLE | 7 
1.1.2 PATH-STAGE-III with P = 0.42. Posteriors = 0.00 1.00 0.00. 
Then: Perform TOTAL-NODAL a 
1.1.3 PATH=STAGE-IV with P = 0.64. Posterfors = 0.00 0.00 1.00. 
Then: Perform MOPP 
1.2 +BMBX with P.-=:0.11. Posteriors = 0.00 0.00 1.00. 
Then: Perform MOPP 


The advantage of this notation is that it can provide more detailed information than the branching tree 


format. 


In its Lisp implementation, the outline display method is somewhat sianclee than the branching tree format. 
The former utilizes a simple depth-first algorithm to traverse the tree in combination will simple text 
formatting program which ensures that lines do not exceed the width of the screen. The latter, because it 
must satisfy more constraints, uses a more complicated algorithm to ensure that a given tree will fit on a 


screen both horizontally and vertically. 


10.8 Additions to the Basic Hodgkins System 


The basic modules of the Hodgkins system acquire the patient description, calculate probabilities of stage, 
and construct, evaluate and display decision trees. A number of features were added to the basic Hodgkins 


disease to perform other tasks associated with the decision analysis. 
10.8.1 Tree Acquisition 


Occasionally, the user will want to evaluate a diagnostic Dim of his own cies to see how it stacks up 


against the nian selected by the program. We provide a a city to allow the user to enter the structure of his 


plan into the computer, so that the stant may be Svaluaied ‘iid displayed song with the pregram’ s ecliona:. 


The program requires the user to enter the plan depth first, using display formats similzr to those used for 
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output of program-generated plans. The program begins by — the user for the first test:in the plan. 
Given the test, and its known pombe fewilbs the Program can display the first level of a diagnostic plan, . 
showing the test and tickaches contespiniding ach oF the rea For exh of the result branches in turn, the 
program asks for the action -- test or treatment -- to take next. If the action isa ; treatment, the branch is 
terminated. Otherwise, the action is a test, so the program recursively asks about each of its results. Such 


questioning continues until all leaves of the tree are filled in. 


To integrate. this facility into the remainder of the decision analysis system, there are program to calculate the 
utility of a prespecified ees Lae Each time pee = are constructed and ara oe the sacs of the 


user-supplied plan is calculated and ranked with the others, 
10.8.2 Sensitivity and Threshold Analysis 


The program contains facilities for performing asa and d threshold analy paoraahd ae is the 
study of the effects of small perturbations of the data on the results of decision esi aa For example, the 
expected utility of doing laparotomy depends on the cost, in terms of morality, of blecting the patient to 


. the operation. Sensitivity analysis can determine what diagnostic plains would be reobenunented for mottality 


estimates ranging below and above the current estimate. 


Sensitivity analysis is implemented as an additional epee program on aad Hodekin disease aye: This 
top-level program varies selected parameters and calculates the optimal visa for ‘eu value: Parametric 
variation may be of two types: numeric and non-numeric. In numeric variation, numeric ‘parameters such a8 
conditional brovabuiess are varied over a te For wre: the conditional propebiiny of High tet 
involvement might be caused to bal from 04 to ee to see what effect tis has on “ eat/(reaument 
decision. In non-numeric variation, the value ofa finding i is caused to omy over all possible wales! to see how 


the finding affects a decision. 
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Appendix I - Comprehensive List of Probabilities for Hodgkins Disease Patieats 


A large amount of information important to oo making i in Hodghins disease has been compiled as the 
basis of the decision: aly computer program. In order to mate this information available to those who ~ 


may not have access to our program, we present the lowing tables of of probabil of stage for: 


1. PRIOR PROBABILITIES based on the three interdependent findings of sex, histological subtype and the 


presence (or absence) of symptoms. 

2. CONDITIONAL PROBABILITIES for finding|stage which are used to modify these prior probabilities. . 
Conditional probabilities are given for both the clinical findings, and the test results. 

3. POSTERIOR PROBABILITIES for all combinations of the clinical. findings. 

N.B. Although probabilities of pathologic stage are given for all histologic subtypes and for all age groups, the 
numbers of patients in the “less than 12 years" age group, and, with the “lymphocyte, predominant” and 
"lymphocyte depleted” histologic subtypes are mnie so probabilities unvolene these pasa have 


considerable uncertainty. It must also be realized that ihhege probabilities apply only to , cupricdiaphiriguiade 


presentations of Hodgkins disease. 


The following abbreviations have been used: 


A = Asymptomatic B = Symptomatic 

NS = Nodular sclerosis MC = Mixed cellularity 

LD = Lymphocyte depleted LP = Lymphocyte predominant 
M= Male | F = Female 

+ = Positive ~ = Negative 

LN = Left cervical lymph nodes RN = Right cervical lymph nodes 


MED = Mediastinum . 
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1. 


NS 
NS 
NS 
NS 
MC 
MC 
MC 
MC 
LP 
LP 
LP 
LP 
LD 
LD 
LD 


LD 


PRIOR PROBABILITIES 


> > & 


FEMALE 
MALE 
FEMALE 
MALE 
FEMALE 
MALE 
FEMALE 
MALE 


FEMALE 


‘MALE 


FEMALE 
MALE 
FEMALE 
MALE 
FEMALE 


MALE 
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- for interdependent findings HISTOLOGY/SYMPTOMS/SEX 


STAGE 
I+] 


0.72 
0.66 
0.66 
0.34 


STAGE 


III 
0.27 
0.29 
0.37 
0.50 
0.36 
0.47 
0.56 
0.44 


0.18 


0.21 


0.41 
0.39 


0.47 


0.43 


0.48 


0.42 


0.16 


0.35 


STAGE 


IV 
0.01 
0.06 
0.08 


0.09 
0.09 


0.04 


0.02 
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0.11 
0.26 
0.13 


0.29 


0.17 


0.36 
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2. CONDITIONAL PROBABILITIES FOR FINDING{ STAGE 


AGE i ee 
Younger than 12 years I+II 0.03 III 0.05 IV 0.08 
12 to 39 years I#dl: 0.84: 5 lid @.74 VE 0546 | 


Older than 39 years ItII 0.14 III 0.23 IV 0.47 


LEFT NECK NODAL INVOLVEMENT 


LN+ I¢il.,0.6832 «4 lit 0.76 IV: 0.73 |: 

LN- I+II 0.32 III 0.26 IV 0.27 
RIGHT NECK NODAL INVOLVEMENT ae a thes 

RN+ Itt]: 0.472. Iti 9.49 IV.9:;70. . 

RN- . I+II 0.63 III 0.61 IV 0.30 
MEDIASTINAL NODAL INVOLVEMENT Cu ae SER 

MED+ ItIT. 0.62: 3 lid 9.63 IV: 0:63 

MED- I+II 0.38 II 0.47 IV 0.47 


SPLEEN SCAN 


Normal I#il..0.93\: 2 id @ 0.64 IV:8, 49 : 
III B 0.66 
7 UER LE oe 48...0 (iat & 
Enlarged >16 cm I+II 0.07 III A.0.28 IV 0.32 
ee seo) olga. B 0.29 Sate 
Filling defects Ii] 0.000: » gg a 0.09 IV: @.41 |. 
III B 0.10 
Enlarged with filling defects 
I+II 0.00 TII A 0.08 IV 0.08 
III B 0.07 
LIVER SCAN 
-ve I+II 0.23 III 0.23 IV 0.66 
+ve I+II 0.77 III 0.77 IV 0.46 
PERCUTANEOUS LIVER BIOPSY 
~ve I+II 1.00 ITI 1.00 Iv 0.80 


+ve I+IT 0.00 III 0.60 IV 0.20 
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LIVER BIOPSY VIA PERITONEOSCOPY 


ve I+II 1.00 III 1.00 IV 0:39 
ius ss T#IT 0.00 1110.00 —«1V 0.61 


BONE MARROW BIOPSY 


-ve I+II 1.00 ~ TIT 1.00 - IVA 0.81 
. Iv'B 0.56 
+ve I+tII 6.00 III 0.00 _IV A 0.19 
IV B 0.44 
GALLIUM SCAN (Abdominal nodes) 

-ve I+II 0.90 —sdII._ A (0.68 IV A 0.61 
. IIT B 0,56 IV B 0.54 
+ve I+II 0.10 III A 0.35 Iv A 0.39 
«TIT B 0.44 IV B 0.46 

LYMPHANG IOGRAM | 
+ve I+II 0.18 | III A 0.66 ‘Iv A 0.60 
III B 0.67 IV B 0.70 
equivocal ItII 0.23 III A 0.16 IV A.0.14 
ae III B 0.12 IV B 0.12 
-ve I+It 0.69 III A 0.20 IV A 0.26 
III B 0.21 IV B 0.18 
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Probability (III) III W. 


ASYMPTOMATIC (A) PATIENTS 


NS M AGE<X12 
LN+RN+MED+ 0.6 0.37 0.13 
LN+RN+MED- 0.41 0.43 0.16 
LN+RN-MED+ 0.56 0.38 0.06 
LN+RN-MED- 0.47 0.46.. 0.07 
LN-RNFMED+ 0.58 0.3 0.12 
LN-RN+MED- 0.49 0.37. 0.16 
LN-RN-MED+ 0.64 0.31 0.05 
NS M 11<AGE<40 
LN+RN+MED+ 0.69 0.27 0.04 
LN+RN+MED- 0.61 0.34 0.05 
. LN+RN-MED+ 0.73 0.26. 0.01 
LN+RN-MED- 0.65 0.33 9.02 
LN-RN+MED+ 0.76 0.21. 0.03 
LN+RN+MED- 0.69 0.27 0.04 
LN-RN-MED+ 0.79 0.2 0.01 
NS M 39<AGE 
LN+RN+MED+ 0.49 0.35 0.16 
LN+RNSMED- 0.4 0.41. 0.19 
LN+RNrMED+ 0.56 0.37 0.07 
LN+RN-MED- 0.47 0.46. 0.08 
LN-RN+MED+ 0.57 0.29 0.15 
LN-RN+MED- 0.47 0.36. 0.18 
LN-RN-MED+ 0.64 0.3 0.06 
NS F AGE<12 
LN+RN+MED+ 0.6 0.36 0.04 
LN+RN+MED- 0.61 0.46 0.06 
LN+RN-MED+ 0.63 0.36 0.02 
LN+RN-MED- 0.54 0.44 0.02 
LN-RN+MED+ 0.67 0.29 0.03 
LN-RN+MED- 0.59 0.37 0.04 
LN-RN-MED+ 0.71 0.28 0.01 
NS F 11<AGE<40 
LN+RN+MED+ 0.75 0.24 0.01 
LN+RN+MED- 0.67 0.31 0.01 
LN+RN-MED+ 0.77 0.23 0.0 
_LN+RN-MED- 0.7 0.3 0.0 
LN-RN+MED+ 0.81 0.18 0.01 
LN-RN+MED- 0.74 0.26 0.01 
LN-RN-MED+ 0.82 0.17 0.0 
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NS F 39<AGE 


LN+RN+MED+ 0.59 0.36 0.05 
LN+RN+MED- 0.5 0.44 " 0.06 
LN+RN-MED+ «0.63 0.35 ». 0.02 
LN+RN-MED- 0.64 0.43 —60..02 
LN-RN+MED+ 0.67 0.29. 0.04 
LN-RN+MED- 0.59 0.36 0.05 
LN-RN-MED+ 0.71 0.28 0.02 
MC M AGE<12. 
LN+RN+MED+ 0.29 0.56 0.21 
LN+RN+MED- 0.22 0.56 0.23 
LN+RN-MED+ 0.35 0.56 — 0.1 
LN+RN-MED- 0.27 0.62 0.11 
LN-RN+MED+ 0.36 0.44 0.2 
LN-RN+MED- 0.28 . 0.49 0.23 
LN-RN-MED+ 0.42 0.48 0.09 
MC M 11<AGE<40 
LN+RN+MED+ 0.48 0.46 0.07 
LN+RN+MED- 0.39 0.62 0.08 
LN+RN-MED+ 0.62 0.45 0.03 
LN+RN-MED- 0.43 0.53 0.03 
LN-RN+MED+ ' 0.67 0.37 0.07 
LN-RN+MED- 0.47 0.46. 0.08 
LN-RN-MED+ 0.61 0.37 0.03 
MC M 39<AGE 
LN+RN+MED+ 0.27 0.47 0.26 
LN+RN+MED- 0.21. 0,62 0.28 
LN+RN-MED+ 0.34 0.54 0.12 
LN+RN-MED- 0.26 0.6 0.13 
LN-RN+MED+ 0.34 0.41... 0.26 
LN-RN+MED- 0.26 0.46. 0.27 
LN-RN-MED+ 0.42 0.47 0.11 
MC F AGE<X12 
LN+RN+MED+ 0.37 0.4 0.23 
LN+RN+MED- 0.29 0.46 0.26 
LN+RN-MED+ 0.45 0.44. 0.11 
LN+RN-MED- 0.36 0.62. 0.12 
LN-RN+MED+ 0.44 0.34 . 0.22 
LN-RN+MED- 0.36 0.39 0.25 
LN-RN-MED+ 0.63 0.37 0.1 
MC F 11<AGE<40 
LN+RN+MED+ 0.59 0.33 0.07 
’ LN+RN+MED- 0.5 0.41. 0.99 
LN+RN-MED+ 0.64 0.33 0.03 
LN+RN-MED- 0.55 0.41 0.04 
LN-RN+MED+ 0.67. 0.27 0.07 
LN-RN+MED- 0.58 0.34 0.08 
LN-RN-MED+ 0.71 0.26 0.03 
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MC F 39<AGE 


LN+RN+MED+ 0.365 0.37 0.28 
LN+RN+MED- 0.27 0.42 © 0.31 
LN+RN-MED+ mae 0.44 0.43 0.18 
LN+RN-MED- 0.36 0.6 . 0.16 
LN-RN+MED+ : 0.42. 0.32 . 0.26 
LN-RN+MED- 0.34 0.36. 0.3 
LN-RN-MED+ 0.52 0.36 0.12 
LP M AGE<12 : 
LN+RN+MED+ 0.57 0.27. . 0.16 
LN+RN+MED- 0.48 0.33. = 0.19 
LN+RN-MED+ 0.65 © 0.28. - 0.07 
. LN+RN-MED- 0.66. 0.36..°° 0.09 
LN-RN+MED+ 0.64 0.22. «| «0.14 
LN-RN+MED- 0.55. 0.27... = 0.18 
LN-RN-MED+ . 0.72. 0.22 - 0.06 
LP M 11<AGE<40 
LN+RN+MED+ 0.76 | 0.19., . 0.04 
LN+RN+MED- 0.69 0.26 ‘° 0.06 
LN+RN-MED+ ; 0.8 . 0.19. . 0.02 
LN+RN-MED- 0.73 0.26 . 0.02 
LN-RN+MED+ ; 0.82 0.16... 0.04 
LN-RN+MED- 0.76. 0.2 0.06 
LN-RN-MED+ 0.85 0.14 0.01 
LP M 39<AGE 
LN+RN+MED+ 0.66 0.26 0.19 
LN+RN+MED- 0.46 0.31 > 0.23 
LN+RN-MED+ 0.64 0.28. °° 0.09 
LN+RN-MED- 0.665 . 0.34°° 0.11 
LN-RN+MED+ , 0.62 0.21°° © 0.17 
LN-RN+MED- «0.63 i286 «i 
LN-RN-MED+ 0.71.. 0.22, 0.08 
LP F AGE<12 
LN+RN+MED+ 0.69 0.26 0.06 
LN+RN+MED- 0.61 0.32... . 0.08 
LN+RN-MED+ 0.73:. 0.24 °° 0.02 
LN+RN-MED- 0.66, . 0.31. > 0.03 
LN-RN+MED+ 0.76. © 0.19, © 0.06 
LN-RN+MED-  , 0.68 = 00.28 0.07 
LN-RN-MED+ -0.79- 0.19-  ° 0.02 
LP F 11<AGE<40 
LN+RN+MED+ 0.83 0.16 0.01 
LN+RN+MED- 0.77. = 0.21, 0.02 
LN+RN-MED+ 0.86 0.16 , 0.01 
LN+RN-MED- “0.79 ° 0.2 °° 0.01 
LN-RN+MED+ 0.87 0.12 ~* 0.01 
LN-RN+MED- 0.82 0.16... 0.02 
LN-RN-MED+ 0.89 0.11 0.0 
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LP F 39<AGE 


LN+RN+MED+ 0.68 0.24 - 0.08 
LN+RN+MED- 0.6 0.31 - 0.1 
LN+RN-MED+ 0.73 0.24 ** ~ 0.08 ss 
LN+RN-MED- ~ 0.66 — 0.31°* 0.04 ! 
LN-RN+MED+ 0.76 0.19" 0.06 
LN-RN+MED- 0.67. 0.24" ~~ 0.08 
LN-RN-MED+ 0.79 0.18 ~ 0.03 
LD M AGE<12 
LN+RN+MED+ 0.14 0.35 0.61 
LN+RN+MED- 0.1 0.36 0.64 
LN+RN-MED+ 0.21 0.49'° ' 0.3 
LN+RN-MED- 0.16 °'0.62° ~* 0.32 
LN-RN+MED+ 0.18 0.31 0.61 
LN-RN+MED- 0.13 0.33 0.64 
LN-RN-MED+ 0.27 0.44 0.29 
LD M 11<AGE<40 
LN+RN+MED+ 0.33 0.43 0.24 
LN+RN+MED- 0.26 0.48 0.27 
LN+RN-MED+ 0.4  0.49°' 0.11 
LN+RN-MED- 0.32 0.56 0.13 
LN-RN+MED+ “0.4 0.37 ° 0.23 
LN-RN+MED- 0.31 0.42 0.26 
LN-RN-MED+ 0.48 0.41 0.11 
LD M 39<AGE 
LN+RN+MED+ 0.12 0.3 © 0.67 
LN+RN+MED- 0.09 0.32 0.6 
LN+RN-MED+ 0.2 0.465 0.36 
LN+RN-MED-~ 0.16 0.48 0.37 
LN-RN+MED+ 0.16 0.27 0.67 
LN-RN+MED- 0.11 0.29- 0.6 
LN-RN-MED+ 0.26 0.4 0.36 
LD F AGE<12 
LN+RN+MED+ 0.26 0.47 0.28 
LN+RN+MED- 0.19 0.61 0.31 
LN+RN-MED+ 0.32 0.66 0.14 
LN+RN-MED- 0.24 0.61 0.16 
LN-RN+MED+ 0.31 0.41 0.28 
LN-RN+MED- 0.24 0.46 © 0.31 
LN-RN-MED+ 0.39 0.48 0.13 
LD F 11<AGE<40 
LN+RN+MED+ 0.46 0.46 0.1 
LN+RN+MED- 0.36 0.62 0.12 
LN+RN-MED+ 0.6 0.46 0.04 
LN+RN-MED- 0.41 0.64 0.065 
LN-RN+MED+ 0.53 0.37 0.1 
LN-RN+MED- 0.44 0.46 0.11 
LN-RN-MED+ 0.68 0.38 0.04 
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LD F 39<AGE 


LN+RN+MED+ 0.23 0.43. 0.34 
LN+RN+HED- 0.17 0.46 , 0.36 
LN+RN-MED+ s 0.31 0.62:- - 0.17 
LN+RN-MED- 0.24; 0.58>, 0.29 
LN-RN+MED+ 0.29.  0.38;- - 6.33 
LN-RN+MED- 0.22 0.42+:.; 0.36 
LN-RN-MED+ 0.38 0.46. 0.16 
SYMPTOMATIC (B) PATIENTS 
WS M AGE<12 
LN+RN+MED+ 0.19 0.47. . 0.34 
LN+RN+MED- 0.145 - 0.6 3: . 0.36 
LN+RN-MED+ 0.26, 0.875. - 0.17 : 
LN+RN-MED- 0.19-. - 0.62-. - 0.18 
LN-RN+MED+ 0.25... 0.42-. + 0.33 
LN-RN+MED- 0.18 0.465 0.36 
LN-RN-MED+ 0.32 0.51 0.17 
NS M 11<AGE<40 x 
LNRNEMED+ 0.38. 0.49 - 0.33 
LN+RN+MED- 0.3. 0.65. 0.16 
LN+RN-MED+ "0.43 - 0.61 . . 0.06 
LN+RN-MED- 0.84 0.88, 0.07 
LN-RN+MED+ 0.46 0.42... - 0.18 
LN-RN+NED- 0.37 0.48 | 0.16 
LN-RN-MED+ 0.62 0.43 0.06 
NS M 39<AGE . 
‘LN+RN+MED+ 0.18 0.43 0.39 
LN+RN+MED- 0.13 0.45. - 0.42 
LN+RN-MED+ 0.25 0.65. . 0.21 
LN+RN-MED- 0.19 0.59. + 0.22 
LN~-RN+MED+ 0.23 0.38 =. - 0.39 
LN-RN+MED- 0.17 0.41 0.42 
LN-RN-MED+ 0.31 0.49 - 0.2 
NS F AGE<12 : a 
LN+RN+MED+ 0.39 0.42... 0.19 
LN+RN+MED- 0.3 . 10.48 3° ; 0.22 
LN+RN-MED+ 0.46 +: 70.48. 50.09 
LN+RN-MED- 20.87... 0.53 3. 40.1 
-LN-RN4MED+ “0.46 «0.36 |. 0.18 43 
LN-RN+MED- 0.37 0.42 0.21 
LN-RN-MED+ 0.54 0.38 0.08 
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NS F 11<AGE<40 


LN+RN+MED+ 0.6 0.34 0.06 
LN+RN+MED- 0.61 0.42 0.07 
LN+RN-MED+ 0.64 0.34 0.02 
LN+RN-MED- 0.55 0.42 0.03 
LN-RN+MED+ 0.67 0.27 0.06 
LN-RN+MED- 0.59 0.35 0.07 
LN-RN-MED+ 0.71 0.27 0.02 
NS F 39<AGE 
LN+RN+MED+ 0.37 0.4 0.23 
LN+RN+MED- 0.29 0.45 0.26 
LN+RN-MED+ 0.45 0.44 0.11 
LN+RN-MED- 0.36 0.62 0.12 
LN-RN+MED+ 0.45 0.34 0.22 
LN-RN+MED- 0.36 0.39 0.26 
LN-RN-MED+ 0.53 0.37 0.1 
MC M AGE<12 F 
LN+RN+MED+ 0.09 0.33 0.58 
LN+RN+MED-" 0.07 0.34 0.6 
LN+RN-MED+ 0.16 0.49 0.36 
LN+RN-MED- 0.11 0.52 0.37 
LN-RN+MED+ 0.12 0.3 0.58 
LN-RN+MED- 0.08 0.31 0.61 
LN-RN-MED+ ~ 0.19 0.45 0.36 
MC M 11<AGE<40 
LN+RN+MED+ 0.24 0.45 0.3 
LN+RN+MED- 0.18 0.49 0.33 
LN+RN-MED+ 0.31 0.64 0.16 
LN+RN-MED- 0.24 0.6 0.16 
LN-RN+MED+ 0.3 0.4 0.3 
LN-RN+MED- 0.23 0.44 0.33 
LN-RN-MED+ 0.38 0.47 0.14 
MC M 39<AGE 
LN+RN+MED+ 0.08 0.28 ' 0.64 
LN+RN+MED- 0.06 0.29 (0.66 
LN+RN-MED+ 0.14 0.46 0.41 
LN+RN-MED- 0.1 0.47 0.43 
LN-RN+MED+ 0.1 0.26 - 0.64 
LN-RN+MED- - 0.07 0.27 0.66 
LN-RN-MED+ 0.18 0.41 0.42 
MC F AGE<12 
LN+RN+MED+ 0.36 0.64 0.1 
LN+RN+MED- 0.28 0.61 0.11 
LN+RN-MED+ 0.4 0.66 0.04 
LN+RN-MED- 0.32 0.64 0.05 
LN-RN+MED+ 0.44 0.47 0.1 
LN-RN+MED- 0.35 0.64 0.11 
LN-RN-MED+ 0.48 0.48 0.04 
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MC F 11<AGE<40 


LN+RN+MED+ 0.54 0.43 0.03 
LN+RN+MED- 0.46 —- 0.52 0.04 
LN+RN-MED+ 0.57 0.42 0.01 
LN+RN-MED- 0.48 0.61 0.01 
LN-RN+MED+ 0.62 0.36 0.03 
LN-RN+MED- 0.53 0.43 0.03 
LN-RN-MED+ « 0.65 0.34 0.01 
MC F 39<AGE 
LN+RN+MED+ 0.36 0.52 0.12 
LN+RN+MED- 0.27 0.59 0.14 
LN+RN-MED+ 0.4 0.56 0.06 
LN+RN-MED- 0.31 0.63 0.06 
LN~-RN+MED+ 0.43 0.45 0.12 
LN-RN+MED- 0.34 0.62 0.14 
LN-RN-MED+ — 0.48 0.47 0.06 
LP M AGE<12 
LN+RN+MED+ 0.18 0.33 0.49 
LN+RN+MED- 0.13 0.36 0.62 
LN+RN-MED+ 0.27 0.46 0.28 
LN+RN-MED- 0.2 0.6 0.3 
LN-RN+MED+ 0.22 0.29 0.48 
LN-RN+MED- 0.17 0.32 - 0.62 
LN-RN-MED+ 0.33 0.4 06.27 
LP M 11<AGE<40 
LN+RN+MED+ 0.39 0.39 0.22 
LN+RN+MED- 0.31 0.44 0.26 
LN+RN-MED+ 0.47 0.43 0.1 
LN+RN-MED- 0.38 — 60.5 0.12 
LN-RN+MED+ 0.47 0.33 0.2 
LN-RN+MED- 0.38 «60.38 0.24 
LN-RN-MED+ 0.55 0.36 0.09 
LP M 39<AGE 
‘LN+RN+MED+ 0.16 0.29 0.55 
LN+RN+MED- ~ 0.12 0.31 _ 0.58 
LN+RN-MED+ 0.25 0:42 ° 9.33 
LN+RN-MED- ~ 9.19 «0.46 0.36 
LN-RN+MED+ 0.2 0.26 0.54 
LN-RN+MED- 0.16. 0.28 0.68 © 
-LN-RN-MED+ 0.31 0.37 0.32 
LP F AGE<12 
LN+RN+MED+ 0.31 0.43 0.26 
LN+RN+MED- 0.23 0.48 0.29 
LN+RN-MED+ 0.38 0.49 © 0.12 
LN+RN-MED- 0.3 ' 0.66 0.14 
LN-RN+MED+ 0.37 - 0.37 0.25 
LN-RN+MED- 0.29 0.42 0.29 
LN-RN-MED+ 0.46 (0.42 0.12 
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LP F 11<AGE<40 


LN+RN+MED+ 0.62 0.39 0.09 
LN+RN+MED- 0.43 0.46 0.11 
LN+RN-MED+ 0.57 0.39 0.04 
LN+RN-MED- 0.48 ~ 0.48 0.05 
LN-RN+MED+ 0.6 0.32 0.08 
LN-RN+MED- 0.51 0.39 0.1 
LN-RN-MED+ 0.66 0.32 0.03 
LP F 39<AGE 
LN+RN+MED+ 0.29 0.4 0.31 
LN+RN+MED- 0.22 0.44 0.34 
LN+RN-MED+ 0.37 0.48 0.15 
LN+RN-MED- 0.29 0.54 0.17 
LN-RN+MED+ 0.35 0.36 0.3 
LN-RN+MED- 0.27 0.39 0.34 
LN-RN-MED+ 0.45 0.41 0.14 
LD M AGE<12 
-LN+RN+MED+ 0.11 0.31 0.58 
LN+RN+MED- - 0.08 0.32 0.6 
LN+RN-MED+ 0.17 0.47 0.36 
LN+RN-MED- 0.12 0.6 0.38 
LN-RN+MED+ 0.13 0.28 0.58 
LN-RN+MED- 0.1 0.3 0.61 
_ LN-RN~MED+ 0.22 0.42 0.36 
LD M 11<AGE<40 
LN+RN+MED+ 0.27 0.43 0.3 
LN+RN+MED- 0.2 0.47 0.33 
LN+RN-MED+ 0.36 ' 0.61 0.16 
LN+RN-MED- 0.27 0.67 0.16 
LN-RN+MED+ 0.34 0.37 0.29 
LN-RN+MED- 0.26 0.42 0.32 
LN-RN-MED+ 0.42 0.44 0.14 
LD M 39<AGE 
LN+RA+MED+ 0.09 0.27 0.64 
LN+RN+MED- 0.07 0.28 0.66 
LN+RN-MED+ 6.16 0.43 0.41 
L.N+RN-MED- ; 0.11 0.45 0.44 
LN-RN+MED+ 0.12 0.24 0.64 
LN-RN+MED- 0.08 0.26 0.66 
LN-RN-MED+ . 0.2 0.38 0.41 
LD F AGE<12 
LN+RN+MED+ 0.2 0.46 0.36 
LN+RN+MED- 0.15 0.48 0.37 
LN+RN-MED+ 0.27 0.56 0.17 
LN+RN-MED- 0.2 0.61 0.19 
LN-RN+MED+ 0.25 0.4 0.34 
LN-RN+MED- 0.19 0.44 0.37 
LN-RN-MED+ 0.34 0.49 0.17 
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LD F 11<AGE<40 


LN+RN+MED+ 0.4 0.47 0.14 
LN+RN+MED- 0.31 0.53 0.15 
LN+RN-MED+ 0.45 0.49 0.06 
LN+RN-MED- 0.36 0.57 0.07 
LN-RN+MED+ 0.47 0.4 0.13 
LN-RN+MED- 0.38 0.47 0.15 
LN-RN-MED+ 0.53 0.41 0.05 
LD F 39<AGE 

LN+RN+MED+ 0.19 0.41 0.4 

LN+RN+MED- 0.14 0.44 0.43 
LN+RN~MED+ 0.26 0.53 0.21 
LN+RN-MED- 0.19 0.58 0.23 
LN-RN+MED+ 0.23 0.37 0.4 

LN-RN+MED- 0.17 0.4 0.43 
LN-RN-MED+ 0.32 0.47 0.21 
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Appendix II - A General Statistical Test For Evaluating Probabilities 


Note: a more extensive discussion of this test, its application and power may be found in Barnett et al, 1981. 


Suppose for a group of N patients numbered from 1 to N according to some neutral procedure (e.g. 
alphabetically), each is known “ be in exactly one of M ‘cucible eee of neath We debi NOM. A 
Bayesian model is used to estimate P,(j), the probability that patient i is in health state j @ = 1,..M). Given 
that the true state of health for each patient is ultimately learned, i might this ae be used to assess 


the validity of the original probabilistic predictions? 
The following procedure allows us to test the null hypothesis Hp that the predictions are accurate. 


1. Consider the model's estimates for P,(j). For patient 1 record P, (1), and for patient 2, P,(2), ...for patient 
M, P,4(M). For patient M+1, we go back to the first stage and record Pyg 410), for patient M+2, 
Puy 422), for patient 2M, Pyy4(M), for patient 2M +1, Pom + 1), etc.. Under this rule, we choose one of 
the predictions made for each patient, and over the entire set, choose roughly an equal number of predictions 


for each of the M states of health. 


2.Take the N predictions just chosen and list them in decreasing order. Let us define the largest as ry, the 


second largest as To, the smallest as Ty; etc. Note that ry is not necessarily associated. with patient number 1. . 


3. Now the patients are to be divided into groups within which the chosen predictions are close together, even 
though, for different people, the predictions may refer to different states of health. For example, if the state 
of health is a stage of Hodgkins disease, one patient may have a 0.4 chance of being in stage +I], while 
another has a 0.4 chance of being in state III. If these predictions are chosen for study in step 1, both patients 


will be grouped together, 
Begin forming a first group with the patient with prediction r1, the patient with prediction rp, etc. Stop, upon 
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21 IV 0.25 III 
23 III 0.26 IV 

28 I+II 0.26 II! 
33 IV 0.26 I+II 
41 III - 0.26. III* 
43 ItII 0.26 I+II* 


Note that, since we are considering three states of health for Hodgkins disease (stage I and II, stage III, and 


stage IV), the use of rule 1 leads to the recorded stages shown for the patients listed. 


Here X = 5, because only the cases with a * meet the stated criterion. In this group all the r’s are 0.25. 


then Q 12 X 0.26 = 3 


and A= (12 X 0.25).0.76 = 2.26 = 1.6 
Thus W = -2/1.6 = -1.33 for the group. 


Under the null hypothesis of correct predictions, the calculated W should have approximately a unit normal 


probability distribution. 


B 

7. Calculate Z from theruleZ= 2 i Wj. where W; is the W-value for the jth group, and B the number of 
i= 

groups formed. If the predictions are accurate then Z would‘ be Chi-distributed with B degrees of freedom. 


One can use this fact to determine the significance level (p-value) of the value of Z calculated from the data. 
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Appendix ITE - Information on Lymphangiogram Used in the Hodgkins Program 


IIL.1 Where Results are Reported as Positive/Equivocal/Negative 


. SOURCE NODES HISTOLOGICALLY POSITIVE NODES HISTOLOGICALLY NEGATIVE 


LAG +ve LAG -ve LAG ? LAG +ve LAG -ve LAG ? 
Aisenberg('71) 8 2 0 5 5 5 
Enright('70) 14 7 8 6 17 14 
Glees(*73):. - 18 2 0. 8. » 20° 2 
Hanks('72) 6 0 0 1 3 0 
Hass('71) 16 0 0 1 27 0 
Jelliffe('70) 6 1 2 4 10 2 
Kaplan(73) 61 5 6 19 - 137 43 
Lowenbraun('70) 4 0 0 3 2 0 
Mitchel1l('72) 10 20 0 11 16 2 
Paglia ('73) 10 4 0 a ee 17 4 
Prosnitz(‘'72) 13 4 2 0 10 7 
Sutcliffe('76) 22 4 2 2 54 6 

187 31 19 67 321 85 


False negative = 31/237 = 13.1% 
False positive = 67/463 = 12.3% 


TH.2 Where Results are Reported.as Positive/Negative Only . 


SOURCE —S—- NODES _HISTOLOGICALLY POSITIVE 


-LAG +ve LAG -ve 
Banfi('74) 29 0 
Castellino('74) 41 0 
Cotman('77) 12 7 
Garcia('71) 2 4 
Hermreck(1976) 7 5 
Hellman('74) 26 2 
Kademian('77) 26 3 
Martire('74) 13 0 
Urlaub('79) 19 9 

174 40 


False negative = 40/214 = 18.7% 


False positive = 81/570 = 14.2% 


Overall adding results from both series: 


False negative = 71/451 = 16.7% 
False positive = 138/1033 = 13.4% 
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NODES HISTOLOGICALLY NEGATIVE 


LAG +ve 
3 

14 

10 

2 

14 

12 


LAG -ve 
70 
111 
69 
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Appendix IV - Annotated Trace of Decision Tree Analysis Program in Operation 


In this example, Expand-Top-Level-Choice-Node is called with: 


tests-left = *( LAG): 

prior = "(.81 .17 02), 
lap-done? 2 NIL 
symptomatology = ‘A 


That is, we are considering an A patient with probability 0.81 of being in stage I+II, 17 of being in stage III, 
and .02 of being in stage IV. Tests other than lymphangiogram (LAG). and laparotomy.(LAP) have either 


been done or are contradicted for other reasons. 


The program trace feature of Lisp has been turned on to show the pattern of function calls which carries out 
the decision tree analysis’ The functions Expand-Top-Level-Choice-Node, Expand-Chance-Node, 
Expand-Choice-Node, Treatment-vs-LAP, Choose-Best-Treatment, and Treatment-E-U have been selected 
for tracing. To save space and complexity, some calls to Choose-Best-Treatment and Treatment-E-U at lower 


levels have been deleted. 


(1 ENTER Expand-Top-Level-Choice-Node ((LAG) (0.81 0.17 0.02))) 


First the program calculates, for each treatment, the expected saad af carrying out that treatment 
immediately, without further testing. 


(1 ENTER Treatment-E-U (TNI (0.81 0.17 0.02))) 
a EXIT Treatment-E-U 0.77) 
=) The utility of immediate total nodal irradiation is 0.77 


(1 ENTER Treatment-E-U (MOPP (0. 81 0.17 0.02))) 
2& EXIT Treatment-E-U 0.68) 
=> The utility of immediate MOPP chemotherapy is 068 


(1 ENTER Treatment-E-U' (EM (0.81 0.17 0.02))) 
_o EXIT Treatment-E-U 0.73) 
=) The utility of immediate extended mantle radiotherapy is 0.73 


Next the program calculates the utility of performing LAP followed by optimal therapy defined by 
the results of LAP. 


(1 ENTER Lap-Plan ((0.81 0.17 0.02))) 
(1 ENTER Expand-Chance-Node (LAP (0.81 0.17 0.02) NIL)) 
(1 ENTER Choose-Best-Treatment ((1.0,0.0 0.0))) 
(1 EXIT Choose-Best-Treatment ((0. 82°. EM))) | 
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(1 ENTER Choose-Best-Treatment ((0.0 1.0°0.0)y)> 
(1 EXIT Choose-Best-Treatment ((0.64 . TNI))) 
(1 ENTER Choose-Best-Treatment ((0.0 0.0 1.0))). 
(1 EXIT Choose-Best-Treatment ((0. 66 ahha 
(1 EXIT Expand- Chance- Node... é 
((0.78 . LAP) utility-data- placesholder 
((I+II 0.81 (1.0 0.0 @,@) ((0.82 . EM))) 
(III 0.17 (0.0 1.0 0.0) ((0.64 .;TNI)}) 
(IV 0.02 (0.0 0.0 1.0) ((0.56 . MOPP)))))) - 
(1 EXIT Lap-Plan : 
((0.78.. LAP) utility-data-place-holder 
((I+II 0.81 (1.0 0.0 0.0) ((0.82 . EM))) 
(IIT, 0.17 (0.0.1.0 0,0) ((0.64 ., TNI)) 
ery’ 0.08 (0.00.0 1:6) ((0.88"; ere hy 
==) The utility of LAP is 0.78 


Finally, the program calculates the utility of performing lymphang iogram. 
> (4 ENTER Expand-Chance-Nede (LAG (0.61. 0.47 0.02):-MEE)). 2 


For the result of LAG being +LAG (lymphangipgram positive for. abdoming] nodes), the progran 
calculates the optimal subsequent plan and its utility. 


(4 ENTER Expand-Chotce-Node ((0.67°0.30°0.06) MIL}) ~~ 
(1 ENTER Treatment-vs-LAP ((0.67 0.38 0. 05))), 
(1 ENTER Lap-P lan :-((0.87 0:36 @.08)} = 
(2 ENTER Expand-Chance-Node (LAP (0. 67 0.38 0. 05) mIL)) 
(2 EXIT Expand-Chance-Node 
((0.73 . LAP) utility-data-place-holder 
((1+II 0.57 (a . 0 AS 0) ((0.82 . EM))) 
(III Q. Bg (0, 0.9). ((9. sf*..s oP) 3993 
(1V 0.0 a. 6° “OY a," 120} '((0-86 = MOPP))3)9) 
(1 EXIT Lap: Py 
((0.73 . vAP) ‘utility- -data=place-holdet” 
((I+II ‘0. §7 (1.0 0.0 0.0) ((0.82 . EM))) 
(IIT 0.38 (0.0 1,0,0.0) ((0.64. sPP4)9)) 
(IV 0.05 (0.0 0. 0-1. 0) ((0.65 . “hope )))) j) 
(1 ENTER Choose-Best-Treatment ((0.57 o. 38 0. 05))). 
(1 EXIT Choose-Best-Treatment® UGS | THT my 
(1 EXIT Treatment-vs-LAP . 
((0.73 . LAP) utility- data- place- cholder 
((I+II 0.57 (1.0 0.0 40 2) {958 82 . cen 
(III 0.38 (0.0 1.0 0.0 84 
(1V 0.06 (0.0 0.0 1. 1). AG 66 , “WPPLIALD) hogs 
(1 EXIT Expand-Choice-Node- - aon pean 
((0.73 . LAP) utility-data-place-holder 
((I+II 0.67 (1.0 0.0 6.0) ((0.82 . eli 
(III.0.38 (0.0 1.0 0.0) ((0.64 . TNI))) 
(IV 0.08 (0-0°0. 0 4.0) ((0.86°. MOPP)))))) 


For the result of LAG being ?LAG Caen abet ag the. program. calculates the pind 
subsequent plan and its utility. 4 


(1 ENTER Expand-Choice-Node ((0.87 0.12 0.0 ary). 
(1 ENTER Treatment-vs-LAP ((0.87 0 ey o> 
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(1 ENTER Lap-Plan ((0.87 0.12 0.01))) 
(2 ENTER Expand-Chance-Node (LAP (0.87 0.12 0. a NIL)) 
(2 EXIT Expand-Chance-Node 
((0.79 . LAP) uy ae data-place-helder 
((I+II 0.87 (1.0 0.0 0.0) ((0,82 ; EM))) 
(III 0.12 (0.0:4.9 0:0). ((0.64: . Hts Re 
(IV 0.01. (0:0 0.0150) (€0555 . MOPP)))))) 
(1 EXIT Lap-Plan ay: 79 . LAP) util ity-data>place-hol der 
(CI+IT 0.87 (1.0 050 0.0) ((0.82 .°EM))):. 
(III 0.12 (0.0 1.0 0.0) ((0.64 .. TNI))) © 
(IV 0.01 (0.0 0.0 1.0) (0.55 .. MOPP))))).). 
(1 ENTER Choose-Best-Treatment: ((0. 67.0,12.0.01))) 
(1 EXIT Choose-Best-Treatment ((8.78:. THI))) 
(1 EXIT Treatment-vs-LAP 
((0.79 . LAP) utility-data- piace- holder . 
((I+II 0.87 (1.00.0 0.0) ((0.82..; EM))) 
(III 0.12 (0.0 1.0 0.0) ((0.64 . “INI))) 
(IV 0.01 (0.0 0.0 1.0) ((0.66 » MOPP)))))) 
(1 EXIT Expand-Choice-Node ((0,79 . LAP) utility-data- place: holder 
((I+II 0.87 (1.0 0.0 0.0) ((0.82°. EM))). bee ge 
(III 0.12 (0.0 1.0 0.0) ((0.64 . TNI))) 
(IV 0.01 (0.0 0.0 1.0) ((0.55 . MOPP)))))) 


For the result of LAG being -LAG (negative lymphangiogram), the program calculates the optimal 
subsequent plan and its utility. 


(1 ENTER Expand-Choice-Node ((0.89 0.10 0.01) NIL)) 
(1 ENTER Treatment-vs-LAP ((0.89 0.10 0.01))) 
(1 ENTER Lap-Plan ((0.89 0.10 0.01))) 
(2 ENTER Expand-Chance-Node (LAP (0.89 0.10 0.01) NIL)) 
(2 EXIT Expand-Chance-Node 
((0.79 . LAP) utility-data-place-holder 
((I+II 0.89 (1.0 0.0 0.0) ((0.82 . EM))) 
(III 0.10 (0.0 1.0 0.0) ((0.64 . TNI))) 
(IV 0.01 (0.0 0.0 1.0) ((0.55 . MOPP)))))) 
(1 EXIT Lap-Plan 
((0.79 . LAP) utility-data-place-holder 
((I+IT 0.89 (1.0 0.0 0.0) ((0.82 . EM))) 
(III 0.10 (0.0 1.0 0.0) ((0.64 . TNI))) 
(IV 0.01 (0.0 0.0 1.0) ((0.56 . MOPP))))) 
(1 ENTER Choose-Best-Treatment ((0.89 0.10 0.0 
(1 EXIT Choose-Best-Treatment ((0.79 . TNI))) 
(1 EXIT Treatment-vs-LAP 
((0.79 . LAP) utility-data-place-holder 
(({I+II 0.89 (1.0 0.0 0.0) ((0.82 . EM))) 
(III 0.10 (0.0 1.0 0.0) ((0.64 . TNI))) 
(IV 0.01 (0.0 0.0 1.0) ((0.55 . MOPP)))) 
(1 EXIT Expand-Choice-Node 
((0.79 . LAP) utility-data-place-holder 
((I+II 0.89 (1.0 0.0 0.0) ((0.82 . EM))) 
(III 0.10 (0.0 1.0 0.0) ((0.64 . TNI))) 
(IV 0.01 (0.0 0.0 1.0) ((0.55 . MOPP)))))) 


») 


The lymphangiogram plan is complete. Regardless of the result of ILAG, as shown in the plan 
fragment below, the best action to take subsequently is laparotomy. Hence, LAG can be omitted 
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as the plan beginning with with LAP will be ‘better in all respects, — 


(1 EXIT Expand-Chance-Node oe 
((0.77 . LAG) utility-data-place-holder =~ 
((+LAG 0.25 (0.67 0.88 0.06) : 
((0.73 . LAP) utility-data-place-holder | 
(CIT 0.67 (1.0 0.0 0.6) ((0.82 . EM})) 
(IIT 0.38:(0.0 1.0 0.0) ((0.:64 :-TWI))) | 
(IV 0.05 (0.0 0.0 1.0) ((0.65- -woPP)})))) 
(7LAG 0.22 (0.87 0.12 8.01) 
((0.79 . LAP). eats data-place- ~holder 
((I+II 0.87 (1.0 0.0 0.0) ((0.82:. EmM))) 
(III 0.12 (0.0 1.0.0.0) ((0.64 “1NI)9) 


(IV 0.01 (0.0 0.0 1.0) (0.65 SWPP) 


(-LAG 0.63 (0.89 0.10°0.01) - 
((0.79 . LAP) utility~data- place-holder 
((I+II 0.89 (1.0°0.0 0:0) ((0.82 < EM))) 
(III 0.10 (0.0 1.6 0.0) ¢(8.64 . TNI))) 
(IV 0.01 (0.0 0.0 1.0) ((0. 66 . MOPP))))))))) 
(1 EXIT Expand-Top-Level-Choice-Node 32) . 
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